Automatic merge from submit-queue
Fix typo for federation/*
**What this PR does / why we need it**:
Increase code readability for this new member in v1.5
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:
Could we develop a typo-fix bot along with a k8s terminology dictionary ?
**Release note**:
```release-note
```
Automatic merge from submit-queue
register batch/jobs to federation-apiserver
register batch/jobs api objects to federation-apiserver
**Release note**:
```release-note
Federation: Add `batch/jobs` API objects to federation-apiserver
```
@quinton-hoole @nikhiljindal @deepak-vij
#34261
This enables testing when the dnsprovider is used externally (with a
mock Route53 API, as we do in kops), and also might be useful for
constructing with a particular client instance with extra options.
Automatic merge from submit-queue
fix typo on federation/pkg/dnsprovider/plugins.go
**What this PR does / why we need it**: Increase code readability
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: just fix one minor mistake as first contribution
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 37468, 36546, 38713, 38902, 38614)
Remove extensions/v1beta1 Job
Fixes https://github.com/kubernetes/kubernetes/issues/32763. This endpoint was deprecated in 1.5 and was planned to be removed in 1.6.
**Release note**:
```release-note
Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator.
```
Automatic merge from submit-queue
Rename root BUILD to BUILD.bazel, and rename build-tools/ back to build/
**What this PR does / why we need it**: undoes #35453. `build-tools/` was intended to be a temporary workaround until https://github.com/bazelbuild/bazel/issues/552 was fixed, which is has been, as of bazel 0.4.1.
**Which issue this PR fixes**: fixes#38126
FYI @thockin
since this is likely to be rebase hell, my list of actions:
```console
$ git mv BUILD BUILD.bazel
$ git commit -a
$ git mv build-tools/ build/
$ sift -l 'build-tools' | xargs sed -i -e 's:build-tools/:build/:g'
$ git checkout HEAD docs/
$ git commit -a
```
Automatic merge from submit-queue (batch tested with PRs 38154, 38502)
Rename "release_1_5" clientset to just "clientset"
We used to keep multiple releases in the main repo. Now that [client-go](https://github.com/kubernetes/client-go) does the versioning, there is no need to keep releases in the main repo. This PR renames the "release_1_5" clientset to just "clientset", clientset development will be done in this directory.
@kubernetes/sig-api-machinery @deads2k
```release-note
The main repository does not keep multiple releases of clientsets anymore. Please find previous releases at https://github.com/kubernetes/client-go
```
Automatic merge from submit-queue
genericapiserver: unify swagger and openapi in config
- make swagger config customizable
- remove superfluous `Config.Enable*` flags for OpenAPI and Swagger.
This is necessary for downstream projects to tweak the swagger spec.
Automatic merge from submit-queue
Register metav1.GetOptions with federation APIs.
PR #37693 missed a spot.
cc @kubernetes/sig-federation @nikhiljindal
Automatic merge from submit-queue (batch tested with PRs 38354, 38371)
Add GetOptions parameter to Get() calls in client library
Ref #37473
This PR is super mechanical - the non trivial commits are:
- Update client generator
- Register GetOptions in batch/v2alpha1 group
Automatic merge from submit-queue (batch tested with PRs 38278, 37770)
Refactor REST storage to use generic defaults
This removes the repetition in the REST storage builders by moving the logic to `restoptions.ApplyOptions`. `registry.StorageWithCacher`/`generic.StorageDecorator` no longer assume that they can build the `keyFunc` for arbitrary objects. `restoptions.ApplyOptions` uses the `registry.Store`'s `KeyFunc` for its call to `generic.StorageDecorator`.
```release-note
Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data.
```
Automatic merge from submit-queue
Add CoreDNS as Federation DNS provider
This PR contains following
1. DNS provider implementation for CoreDNS with etcd as backend
2. Script for optionally deploying CoreDNS in federation control plane
Still to do:
1. HA for CoreDNS service need to be handled, would be taken up in subsequent PR
2. Need to configure CoreDNS as fallback DNS to in-cluster KubeDNS(Manual Step). Need to consider automation.
Request for review: @quinton-hoole @madhusudancs @nikhiljindal @kshafiee @deepak-vij
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue (batch tested with PRs 36071, 32752, 37998, 38350, 38401)
Pass addressable values to DeepCopy
Extracted from https://github.com/kubernetes/kubernetes/pull/35728
These are the places we are currently calling DeepCopy incorrectly, and we need to fix, even if we don't pick up the changes to DeepCopy in #35728:
* creating a new cloner means we have no generated functions registered
* passing non-addressable values doesn't pick up generated deep copy functions, and forces us into reflective mode
Automatic merge from submit-queue
[Federation] Implement dry run support in kubefed init
This one implements one of the TODO items pending in the previous set of kubefed PRs.
This one is done on top of another todo PR https://github.com/kubernetes/kubernetes/pull/36310 which is being reviewed separately.
Please review only the last 2 commits in this one.
The design doc PR for kubefed is at https://github.com/kubernetes/kubernetes/pull/34484.
cc @kubernetes/sig-cluster-federation @madhusudancs @nikhiljindal
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
[Federation] `kubefed init` now supports dry run mode.
```
Automatic merge from submit-queue
[Federation] Make federation etcd PVC size configurable
This one implements one of the many TODO items pending in the previous set of kubefed PRs.
The design doc PR is at https://github.com/kubernetes/kubernetes/pull/34484
cc @kubernetes/sig-cluster-federation @madhusudancs
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
[Federation] kubefed init now has a new flag, --etcd-pv-capacity, which can be used to configure the persistent volume capacity for etcd.
```
Automatic merge from submit-queue (batch tested with PRs 37032, 38119, 38186, 38200, 38139)
Detect long-running requests from parsed request info
Follow up to https://github.com/kubernetes/kubernetes/pull/36064
Uses parsed request info to more tightly match verbs and subresources
Removes regex-based long-running request path matching (which is easily fooled)
```release-note
The --long-running-request-regexp flag to kube-apiserver is deprecated and will be removed in a future release. Long-running requests are now detected based on specific verbs (watch, proxy) or subresources (proxy, portforward, log, exec, attach).
```
Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)
remove unnecessary fields from genericapiserver config
Cleans up some unnecessary fields in the genericapiserver config.
This removes all dependencies on Config during cert generation, only operating
on ServerRunOptions. This way we get rid of the repeated call of Config.Complete
and cleanly stratify the GenericApiServer bootstrapping.
Automatic merge from submit-queue
Create GetOptions to allow passing "ResourceVersion" to GET requests.
Ref #37473
@kubernetes/sig-api-machinery @smarterclayton
Automatic merge from submit-queue
Enable updates in federated ingress controller tests
And make channel in test helper non-blocking.
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Reuse fields and labels
This should significantly reduce memory allocations in apiserver in large cluster.
Explanation:
- every kubelet is refreshing watch every 5-10 minutes (this generally is not causing relist - it just renews watch)
- that means, in 5000-node cluster, we are issuing ~10 watches per second
- since we don't have "watch heartbets", the watch is issued from previously received resourceVersion
- to make some assumption, let's assume pods are evenly spread across pods, and writes for them are evenly spread - that means, that a given kubelet is interested in 1 per 5000 pod changes
- with that assumption, each watch, has to process 2500 (on average) previous watch events
- for each of such even, we are currently computing fields.
This PR is fixing this problem.
Automatic merge from submit-queue
Remove ExportOptions from api/internal and use unversioned
Should only have one internal object in use
Part of #37530
Automatic merge from submit-queue
move parts of the mega generic run struct out
This splits the main `ServerRunOptions` into composeable pieces that are bindable separately and adds easy paths for composing servers to run delegating authentication and authorization.
@sttts @ncdc alright, I think this is as far as I need to go to make the composing servers reasonable to write. I'll try leaving it here
Automatic merge from submit-queue
Fix package aliases to follow golang convention
Some package aliases are not not align with golang convention https://blog.golang.org/package-names. This PR fixes them. Also adds a verify script and presubmit checks.
Fixes#35070.
cc/ @timstclair @Random-Liu
Automatic merge from submit-queue
Fix infinte loop in federated ingress controller
Previously ingress controller was constantly scheduling reconcilation, even if no updates were needed. That behavior creates a big mess in the logs and consumes resources.
This pr also fixes the stop function for federated ingress controller.
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Revision handling in federated deployment controller
Deployment controller in regular kubernetes automatically adds an annotation in deployment. This causes a bit of confusion in controller and tests. This PR skips revision annotation in checks. In the next K8S release we will need to have better support for deployment revisions.
Helps with #36588
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Stop deleting underlying services when federation service is deleted
Fixes https://github.com/kubernetes/kubernetes/issues/36799
Fixing federation service controller to not delete services from underlying clusters when federated service is deleted.
None of the federation controller should do this unless explicitly asked by the user using DeleteOptions. This is the only federation controller that does that.
cc @kubernetes/sig-cluster-federation @madhusudancs
```release-note
federation service controller: stop deleting services from underlying clusters when federated service is deleted.
```
Automatic merge from submit-queue
Fixing the logic to select first cluster in federated ingress controller
Ref https://github.com/kubernetes/kubernetes/issues/36074.
Before this change, ingress controller was using cluster with clusterIndex = 0 as the first cluster to create the ingress in.
But the ordering of clusters can change and hence ingress controller ended up creating the ingress in multiple clusters.
This PR fixes it by using an annotation on federated ingress. Controller now picks up a cluster randomly as the first cluster and creates ingress there. This clusters name is stored as an annotation on the federated ingress. Contoller does not create an ingress in any other cluster if this annotation is set on the federated ingress and IP has not been propagated. Once IP has been propagated, controller creates the ingress in all clusters.
cc @kubernetes/sig-cluster-federation @madhusudancs
Automatic merge from submit-queue
Fix doc links in Federation readme
**What this PR does / why we need it**:
The user guide and admin guide links were swapped round
**Release note**: NONE
Automatic merge from submit-queue
populate ready replicas and aviable replicas to federated replicaset …
populate ready replicas and aviable replicas to federated replicaset status
@nikhiljindal #33312#29481#32771
@deepak-vij
Automatic merge from submit-queue
Fixing the ingress annotation key in federated ingress controller
Fixing a few issues found during debugging https://github.com/kubernetes/kubernetes/issues/36074.
cc @kubernetes/sig-cluster-federation @madhusudancs @bprashanth
Automatic merge from submit-queue
[Federation][join-flags] Add flags for cluster context and secret names while joining clusters to federation.
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make
`kubefed join` consistent with `kubefed unjoin`. This also means
`--cluster-context` is now made a flag and defaults to cluster name if
unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Fixes: Issue #35954
cc @kubernetes/sig-cluster-federation @quinton-hoole @irfanurrehman
Automatic merge from submit-queue
Fields with omitempty tag should still be considered as optional
We've added an "+optional" tag while ago for optional fields. Before that OpenAPI spec generated assumed all fields with "omitempty" in their json tags are optional. This should be still the case (as well as +optional tag) until these two things happen:
- We update all documentation asking developers to use +optional (My bad, I should have added this after the +optional PR)
- We fix swagger 1.2 spec generator to use +optional instead of omitempty.
Fixes#37149
Automatic merge from submit-queue
Make desired objects in fed controller reconcilation function fully writable
When creating desired objects for sub-cluster in reconciliate functions we don't make full copies. This may result in unexpected race condition in the future when someone actually adds some object modifications (not needed in most cases but anyway - the cost of extra copy will be relatively small comparing to inter-cluster traffic). In case of ReplicaSet and Deployment the spec copy was a shallow one - ok for setting just replicas but also error-prone in the future.
cc: @quinton-hoole @nikhiljindal @madhusudancs
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make `kubefed
join` consistent with `kubefed unjoin`. This also means `--cluster-
context` is now made a flag and defaults to cluster name if unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Automatic merge from submit-queue
[Federation][Bugfix] Make the dns provider configurable in kubefed init
Fixes: Issue #36690
The design doc PR relating the whole feature for kubefed is at #34484.
cc @kubernetes/sig-cluster-federation @madhusudancs @nikhiljindal
Automatic merge from submit-queue
Federation test util fix - 1
* Make nil and empty map equivalent when testing ObjectMeta equivalence
* Add a function that waits for a particular content inside federated store
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Ensure proper serialization of updates and creates in federation test watcher
Fix for finalizer test problems. The issue there was that the updates were coming out of order. It was caused by firing updates in new goroutines in test watcher. The proper solution is to order them in a queue and fire in order on a single goroutine.
Ref: https://github.com/kubernetes/kubernetes/issues/36473#issuecomment-259605870
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Adding cascading deletion support to federation replicaset and deployments
Forked from https://github.com/kubernetes/kubernetes/pull/36330
Ref #33612
Adding cascading deletion support for federated replicasets and deployments.
```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated replicasets and deployments. Setting it to false while deleting a federated replicaset or deployment also deletes the corresponding resource from all registered clusters.
```
Automatic merge from submit-queue
Adding cadcading deletion support for federated secrets
Ref https://github.com/kubernetes/kubernetes/issues/33612
Adding cascading deletion support for federated secrets.
The code is same as that for namespaces. Just ensuring that DeletionHelper functions are called at right places in secret_controller.
Also added e2e tests.
cc @kubernetes/sig-cluster-federation @caesarxuchao
```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated secrets. Setting it to false while deleting a federated secret also deletes the corresponding secrets from all registered clusters.
```
If we have a public & private zone with the same name (which is common
on AWS), this means we can still create records. Also tighten up some
of the logic to allow for zones with duplicate names.
Automatic merge from submit-queue
Adding more e2e tests for federated namespace cascading deletion and fixing bugs
Ref https://github.com/kubernetes/kubernetes/issues/33612
Adding more e2e tests for testing cascading deletion of federated namespace.
New tests are now verifying that cascading deletion happen when DeletionOptions.OrphanDependents=false and it does not happen when DeleteOptions.OrphanDependents=true.
Also updated deletion helper to always add OrphanFinalizer. generic registry will remove it if DeleteOptions.OrphanDependents=false. Also updated namespace registry to do the same.
We need to add the orphan finalizer to keep the orphan by default behavior. We assume that its dependents are going to be orphaned and hence add that finalizer. If user does not want the orphan behavior, he can do so using DeleteOptions and then the registry will remove that finalizer.
cc @kubernetes/sig-cluster-federation @caesarxuchao @derekwaynecarr
Automatic merge from submit-queue
[Federation][init-10c] Fix hyperkube binary path in the image and disable auth mechanisms we don't support for now.
Please review only the last commit here. This is based on PR #36047 which will be reviewed independently.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @nikhiljindal
Automatic merge from submit-queue
Read the federation controller manager kubeconfig from a filesystem path
This decoupling from the Kubernetes API allows admins to run federation control plane components wherever they like, even outside Kubernetes. This also makes the federation controller manager read its config from one single place in a uniform and/or consistent way, instead of spreading the config around command line flags and secrets.
``` release-note
Federation controller manager can consume the federation API server kubeconfig from a file configured via --kubeconfig flag.
If you are upgrading your Cluster Federation components from v1.4.x, please update your `federation-apiserver` and `federation-controller-manager` manifests to the new version:
```
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Switch federated secret controller to use NamespacedName
To make it cleaner and consistent with other controllers.
cc: @quinton-hoole
Automatic merge from submit-queue
[Federation][init-02] Stop passing command factory to the init command.
Please review only the last commit here. This is based on PR #35856 which will be reviewed independently.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @nikhiljindal
Automatic merge from submit-queue
[Federation] Add unit tests for `kubefed init`'s certificate generator.
Please review only the last commit here. This is based on PR #35594 which will be reviewed independently.
These are a subset of unit tests for code introduced in PR #35594
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole
Automatic merge from submit-queue
[Federation][init] Implement `kubefed init` command that performs federation control plane bootstrap.
Please review only the last commit here. This is based on PR #35593 which will be reviewed independently.
I am intentionally not including the unit tests in this PR to better distribute and parallelize reviews. This PR is already big.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
deads2k