Automatic merge from submit-queue
Move post processing/backward compatibility of openapi out of generic package
Post processing step was put into generic package and was the same for both k8s api server and federation api server. They have different backward compatibility list of types. This PR move that step out of generic package and put it in each server's file.
Automatic merge from submit-queue (batch tested with PRs 37617, 40197)
dnsprovider: Add direct access to Route53 data
This methods allow us to develop advanced functionality for Route53,
before we add all the functionality to the cross-provider interface.
Use of these methods should be avoided, and adding methods to the
cross-provider interfaces should be preferred.
```release-note
NONE
```
Automatic merge from submit-queue
Use full package path for definition name in OpenAPI spec
We were using short package name (last part of package name) plus type name for OpenAPI spec definition name. That can result in duplicate names and make the spec invalid. To be sure we will always have unique names, we are going to use full package name as definition name. Also "x-kubernetes-tag" custom field is added to definitions to list Group/Version/Kind for the definitions that has it. This will help clients to discover definitions easier.
Lastly, we've added a reference from old definition names to the new ones to keep backward compatibilities. The list of old definitions will not be updated.
**Release note**:
- Rename OpenAPI definition names to type's full package names to prevent duplicates
- Create OpenAPI extension "x-kubernetes-group-version-kind" for definitions to store Group/Version/Kind
- Deprecate old definition names and create a reference to the new definitions. Old definitions will be removed in the next release.
Automatic merge from submit-queue
error strings should not end with punctuation
**What this PR does / why we need it**:
Delete the end punctuation of error strings
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
https://github.com/golang/go/wiki/CodeReviewComments#error-strings
**Release note**:
```release-note
```
Signed-off-by: yupeng <yu.peng36@zte.com.cn>
Automatic merge from submit-queue
remove unneeded storage options
Cleanup of some storage serialization options that only kube and federation api server (and maybe not even that one) need.
You may have called it a snip, but this moves the options out of generic entirely.
Automatic merge from submit-queue (batch tested with PRs 40638, 40742, 40710, 40718, 40763)
move client/record
An attempt at moving client/record to client-go. It's proving very stubborn and needs a lot manual intervention and near as I can tell, no one actually gets any benefit from the sink and source complexity it adds.
@sttts @caesarchaoxu
Automatic merge from submit-queue (batch tested with PRs 40638, 40742, 40710, 40718, 40763)
move pkg/storage to apiserver
Mechanical move of `pkg/storage` (not sub packages) to `k8s.io/apiserver`.
@sttts
Automatic merge from submit-queue
Fix kubectl->kubefed typo
**What this PR does / why we need it**: The kubefed CLI too had a typo in it's example text.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
Automatic merge from submit-queue
add deads2k to some cmd packages
Selectively adding myself to cmd packages for commands that I've authored or significantly touched.
Automatic merge from submit-queue (batch tested with PRs 40392, 39242, 40579, 40628, 40713)
[Federation][kubefed] Create a dedicated service account for federation controller manager in the host cluster and give it appropriate permissions.
Ref: Issue #39555
cc @kubernetes/sig-federation-pr-reviews @kubernetes/sig-auth-misc @kubernetes/sig-auth-pr-reviews
```release-note
kubefed init creates a service account for federation controller manager in the federation-system namespace and binds that service account to the federation-system:federation-controller-manager role that has read and list access on secrets in the federation-system namespace.
```
Automatic merge from submit-queue (batch tested with PRs 40691, 40551, 40683, 40700, 40702)
Replace + with _ in the image tag version since + is not a valid image tag character.
We already push the image to the registry by replacing `+` with `_` in the
preceding build and push stages. This change is just propagating the same
to the deploy stage.
cc @kubernetes/sig-federation-pr-reviews @nikhiljindal @shashidharatd
Automatic merge from submit-queue (batch tested with PRs 40691, 40551, 40683, 40700, 40702)
Enable protobufs as storage format by default
**Release note**:
```release-note
Change default storage format to protobufs. With this PR apiserver will be writing objects serialized as protobufs to etcd. If the apiserver was upgraded in the existing clusters, until an object will be written, it will still be stored as JSON. Apiserver can deal with some data being in json and some in protobuf format as of 1.4 release.
```
@kubernetes/sig-api-machinery-misc @kubernetes/sig-api-machinery-pr-reviews
We already push the image to the registry by replacing + with _ in the
previous build and push stage. This change is just propagating the same
to the deploy stage.
Although Google Cloud DNS requires strict add & remove calls, most
dnsproviders actually support upsert, and an add & remove is much more
expensive (primarily because of the need to fetch the pre-image).
Add support for 'upsert' operations, which don't require the pre-image,
and simply overwrite the existing record. This is much cheaper on
Amazon Route53, for example.
Automatic merge from submit-queue
Use a wrapper script to locate kubefed and kubectl binaries instead of directly constructing their paths.
This fixes the e2e failures that is now switched to using kubefed.
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 40543, 39999)
remove pkg/feature dependency from genericapiserver.
@sttts I think you just forgot this when you were snipping the rest of the link. If I understood the previous pull correctly, this ought to ensure proper registration of the bits we need, right?
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
move the discovery and dynamic clients
Moved the dynamic client, discovery client, testing/core, and testing/cache to `client-go`. Dependencies on api groups we don't have generated clients for have dropped out, so federation, kubeadm, and imagepolicy.
@caesarxuchao @sttts
approved based on https://github.com/kubernetes/kubernetes/issues/40363
Automatic merge from submit-queue
Added shortName to meta.v1.APIResource.
The shortNames would be used to allow kubectl to have short names representations of resources from aggregated API servers.
```release-note
Adds `shortNames` to the `APIResource` from discovery which is a list of recommended shortNames for clients like `kubectl`.
```
**What this PR does / why we need it**:
The shortNames would be used to allow kubectl to have short names representations of resources from aggregated API servers.
**Special notes for your reviewer**:
I have followed the steps described here [Changing the internal structures](https://github.com/kubernetes/community/blob/master/contributors/devel/api_changes.md)
I have run the following scripts:
hack/update-codegen.sh
hack/update-generated-protobuf.sh
hack/update-codecgen.sh
hack/update-openapi-spec.sh
hack/update-swagger-spec.sh
hack/update-federation-openapi-spec.sh
hack/update-api-reference-docs.sh
See also:
#[38755](https://github.com/kubernetes/kubernetes/pull/38755)
#[38835](https://github.com/kubernetes/kubernetes/pull/38835#event-914403188)
**Release note**:
Extended meta.v1.APIResource by a list of short names. The short names would be used to allow kubectl to have short names representations of resources from aggregated API servers.
Automatic merge from submit-queue (batch tested with PRs 39538, 40188, 40357, 38214, 40195)
genericapiserver: cut off more dependencies – episode 2
Compare commit subjects.
approved based on #40363
Automatic merge from submit-queue
dnsprovider: Add IsEmpty method
When batching changes, it is often handy to know whether a changeset
IsEmpty, and thus does not need to be Apply-ed.
```release-note
NONE
```
Automatic merge from submit-queue
Add version to OpenAPI spec
OpenAPI is missing a version string. It should follow kubernetes version. It is a bugfix that need to be cherrypicked in 1.5 too.
Automatic merge from submit-queue
Optional configmaps and secrets
Allow configmaps and secrets for environment variables and volume sources to be optional
Implements approved proposal c9f881b7bb
Release note:
```release-note
Volumes and environment variables populated from ConfigMap and Secret objects can now tolerate the named source object or specific keys being missing, by adding `optional: true` to the volume or environment variable source specifications.
```
Automatic merge from submit-queue
Remove federated-image.tag from the .gitignore file.
This generated version metadata file was being written to a source
directory and caused a lot of pain. We are moving to a world where
this file is generated in the build/output artifacts directory and
also possibly looking at ways to entirely remove the federation
specific versions file. This is in-line with that goal of removing
the dependency on federated-image.tag file.
cc @kubernetes/sig-federation-misc
Automatic merge from submit-queue
Improve TerminationMessagePath to be more flexible
* Support `terminationMessagePolicy: fallbackToLogsOnError` which allows pod authors to get useful information from containers as per kubernetes/community#154
* Set an upper bound on the size of the termination message path or log output to prevent callers from DoSing the master
* Add tests for running as root, non-root, and for the new terminationMessagePolicy cases.
I set the limit to 4096 bytes, but this may be too high for large pod containers. Probably need to set an absolute bound, i.e. max message size allowed is 20k total, and we truncate if we're above that limit.
Fixes#31839, #23569
```release-note
A new field `terminationMessagePolicy` has been added to containers that allows a user to request `FallbackToLogsOnError`, which will read from the container's logs to populate the termination message if the user does not write to the termination message log file. The termination message file is now properly readable for end users and has a maximum size (4k bytes) to prevent abuse. Each pod may have up to 12k bytes of termination messages before the contents of each will be truncated.
```
These files have been created lately, so we don't have much information
about them anyway, so let's just:
- Remove assignees and make them approvers
- Copy approves as reviewers
Automatic merge from submit-queue (batch tested with PRs 40232, 40235, 40237, 40240)
move listers out of cache to reduce import tree
Moving the listers from `pkg/client/cache` snips links to all the different API groups from `pkg/storage`, but the dreaded `ListOptions` remains.
@sttts
Automatic merge from submit-queue (batch tested with PRs 40187, 40231, 40152)
Update client/server cert generation utilities
Limit generated cert usage to client or server use.
Automatic merge from submit-queue (batch tested with PRs 36693, 40154, 40170, 39033)
make client-go authoritative for pkg/client/restclient
Moves client/restclient to client-go and a util/certs, util/testing as transitives.
Automatic merge from submit-queue (batch tested with PRs 40168, 40165, 39158, 39966, 40190)
Read the VERSIONS file for the FEDERATION_IMAGE_TAG value.
This is particularly important in the local builds where there is no $KUBERNETES_RELEASE defined.
cc @kubernetes/sig-federation-misc @shashidharatd
This generated version metadata file was being written to a source
directory and caused a lot of pain. We are moving to a world where
this file is generated in the build/output artifacts directory and
also possibly looking at ways to entirely remove the federation
specific versions file. This is in-line with that goal of removing
the dependency on federated-image.tag file.
This methods allow us to develop advanced functionality for Route53,
before we add all the functionality to the cross-provider interface.
Use of these methods should be avoided, and adding methods to the
cross-provider interfaces should be preferred.
Automatic merge from submit-queue (batch tested with PRs 39446, 40023, 36853)
Create environment variables from secrets
Allow environment variables to be populated from entire secrets.
**Release note**:
```release-note
Populate environment variables from a secrets.
```
Automatic merge from submit-queue
move pkg/fields to apimachinery
Purely mechanical move of `pkg/fields` to apimachinery.
Discussed with @lavalamp on slack. Moving this an `labels` to apimachinery.
@liggitt any concerns? I think the idea of field selection should become generic and this ends up shared between client and server, so this is a more logical location.
Automatic merge from submit-queue
make client-go more authoritative
Builds on https://github.com/kubernetes/kubernetes/pull/40103
This moves a few more support package to client-go for origination.
1. restclient/watch - nodep
1. util/flowcontrol - used interface
1. util/integer, util/clock - used in controllers and in support of util/flowcontrol
Automatic merge from submit-queue
Build release tars using bazel
**What this PR does / why we need it**: builds equivalents of the various kubernetes release tarballs, solely using bazel.
For example, you can now do
```console
$ make bazel-release
$ hack/e2e.go -v -up -test -down
```
**Special notes for your reviewer**: this is currently dependent on 3b29803eb5, which I have yet to turn into a pull request, since I'm still trying to figure out if this is the best approach.
Basically, the issue comes up with the way we generate the various server docker image tarfiles and load them on nodes:
* we `md5sum` the binary being encapsulated (e.g. kube-proxy) and save that to `$binary.docker_tag` in the server tarball
* we then build the docker image and tag using that md5sum (e.g. `gcr.io/google_containers/kube-proxy:$MD5SUM`)
* we `docker save` this image, which embeds the full tag in the `$binary.tar` file.
* on cluster startup, we `docker load` these tarballs, which are loaded with the tag that we'd created at build time. the nodes then use the `$binary.docker_tag` file to find the right image.
With the current bazel `docker_build` rule, the tag isn't saved in the docker image tar, so the node is unable to find the image after `docker load`ing it.
My changes to the rule save the tag in the docker image tar, though I don't know if there are subtle issues with it. (Maybe we want to only tag when `--stamp` is given?)
Also, the docker images produced by bazel have the timestamp set to the unix epoch, which is not great for debugging. Might be another thing to change with a `--stamp`.
Long story short, we probably need to follow up with bazel folks on the best way to solve this problem.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 34763, 38706, 39939, 40020)
prevent anonymous auth and allow all
https://github.com/kubernetes/kubernetes/pull/38696 for master
@kubernetes/sig-auth
```release-note
Anonymous authentication is now automatically disabled if the API server is started with the AlwaysAllow authorizer.
```
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)
move api/errors to apimachinery
`pkg/api/errors` is a set of helpers around `meta/v1.Status` that help to create and interpret various apiserver errors. Things like `.NewNotFound` and `IsNotFound` pairings. This pull moves it into apimachinery for use by the clients and servers.
@smarterclayton @lavalamp First commit is the move plus minor fitting. Second commit is straight replace and generation.
Automatic merge from submit-queue (batch tested with PRs 39807, 37505, 39844, 39525, 39109)
Made cache.Controller to be interface.
**What this PR does / why we need it**:
#37504
Automatic merge from submit-queue (batch tested with PRs 39475, 38666, 39327, 38396, 39613)
Fix a bug in cascading deletion of federation objects
When FinalizerOrphan is present and set to true in federated object we are currently removing the FinalizerOrphan first and then removing FinalizerDeleteFromUnderlyingClusters. if a reconciliation is triggered in between the finalizer removals, it has undesired effect of object deletion in federated cluster.
So we should remove FinalizerDeleteFromUnderlyingClusters first and then remove FinalizerOrphan, when FinalizerOrphan is set to true.
@nikhiljindal, @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 39475, 38666, 39327, 38396, 39613)
Add checking the nil value
**What this PR does / why we need it**:
Add checking the nil value.
Thanks.
**Special notes for your reviewer**:
Automatic merge from submit-queue (batch tested with PRs 39475, 38666, 39327, 38396, 39613)
Create k8s.io/apimachinery repo
Don't panic.
The diff is quite large, but its all generated change. The first few commits are where are all the action is. I built a script to find the fanout from
```
k8s.io/kubernetes/pkg/apimachinery/registered
k8s.io/kubernetes/pkg/runtime/serializer
k8s.io/kubernetes/pkg/runtime/serializer/yaml
k8s.io/kubernetes/pkg/runtime/serializer/streaming
k8s.io/kubernetes/pkg/runtime/serializer/recognizer/testing
```
It copied
```
k8s.io/kubernetes/pkg/api/meta
k8s.io/kubernetes/pkg/apimachinery
k8s.io/kubernetes/pkg/apimachinery/registered
k8s.io/kubernetes/pkg/apis/meta/v1
k8s.io/kubernetes/pkg/apis/meta/v1/unstructured
k8s.io/kubernetes/pkg/conversion
k8s.io/kubernetes/pkg/conversion/queryparams
k8s.io/kubernetes/pkg/genericapiserver/openapi/common - this needs to renamed post-merge. It's just types
k8s.io/kubernetes/pkg/labels
k8s.io/kubernetes/pkg/runtime
k8s.io/kubernetes/pkg/runtime/schema
k8s.io/kubernetes/pkg/runtime/serializer
k8s.io/kubernetes/pkg/runtime/serializer/json
k8s.io/kubernetes/pkg/runtime/serializer/protobuf
k8s.io/kubernetes/pkg/runtime/serializer/recognizer
k8s.io/kubernetes/pkg/runtime/serializer/recognizer/testing
k8s.io/kubernetes/pkg/runtime/serializer/streaming
k8s.io/kubernetes/pkg/runtime/serializer/versioning
k8s.io/kubernetes/pkg/runtime/serializer/yaml
k8s.io/kubernetes/pkg/selection
k8s.io/kubernetes/pkg/types
k8s.io/kubernetes/pkg/util/diff
k8s.io/kubernetes/pkg/util/errors
k8s.io/kubernetes/pkg/util/framer
k8s.io/kubernetes/pkg/util/json
k8s.io/kubernetes/pkg/util/net
k8s.io/kubernetes/pkg/util/runtime
k8s.io/kubernetes/pkg/util/sets
k8s.io/kubernetes/pkg/util/validation
k8s.io/kubernetes/pkg/util/validation/field
k8s.io/kubernetes/pkg/util/wait
k8s.io/kubernetes/pkg/util/yaml
k8s.io/kubernetes/pkg/watch
k8s.io/kubernetes/third_party/forked/golang/reflect
```
The script does the import rewriting and gofmt. Then you do a build, codegen, bazel update, and it produces all the updates.
If we agree this is the correct approach. I'll create a verify script to make sure that no one messes with any files in the "dead" packages above.
@kubernetes/sig-api-machinery-misc @smarterclayton @sttts @lavalamp @caesarxuchao
`staging/prime-apimachinery.sh && hack/update-codegen.sh && nice make WHAT="federation/cmd/federation-apiserver/ cmd/kube-apiserver" && hack/update-openapi-spec.sh && hack/update-federation-openapi-spec.sh && hack/update-codecgen.sh && hack/update-codegen.sh && hack/update-generated-protobuf.sh && hack/update-bazel.sh`
Automatic merge from submit-queue (batch tested with PRs 39714, 39646)
use etcd2 as storage-backend for federation until federation features are completely tested with etcd3
**What this PR does / why we need it**: move federation etcd to etcd3
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39594
**Special notes for your reviewer**: here is the [link](https://github.com/kubernetes/features/issues/44#issuecomment-270772674) to announcement making etcd3 as default
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 39230, 39718)
[Federation] Kubefed init verifies if control plane pods are up before returning success
This PR updates the functionality as needed in issue https://github.com/kubernetes/kubernetes/issues/37841.
cc @kubernetes/sig-cluster-federation @nikhiljindal @madhusudancs @shashidharatd
Automatic merge from submit-queue (batch tested with PRs 38212, 38792, 39641, 36390, 39005)
Updating federated service controller to support cascading deletion
Ref https://github.com/kubernetes/kubernetes/issues/33612
Service controller is special than other federation controllers because it does not use federatedinformer and updater to sync services (it was written before we had those frameworks).
Updating service controller code to instantiate these frameworks and then use deletion helper to perform cascading deletion.
Note that, I havent changed the queuing logic in this PR so we still dont use federated informer to manage the queue. Will do that in the next PR.
cc @kubernetes/sig-federation-misc @mwielgus @quinton-hoole
```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated services. Setting it to false while deleting a federated service also deletes the corresponding services from all registered clusters.
```
Automatic merge from submit-queue (batch tested with PRs 39466, 39490, 39527)
bump gengo to latest
bumping gengo to limit surprises while working on https://github.com/kubernetes/kubernetes/pull/39475
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 39466, 39490, 39527)
Generate OpenAPI definition for inlined types
Currently OpenAPI definition generator ignores any type's member with an empty json name tag. However, most (if not all) of these types also have "inline" json tag. That means we should inline their members into parent type's OpenAPI definition instead of ignoring them. This resulted in many types missing common parameters such as "Name". It look something serious to me for OpenAPI spec and I suggest we merge and cherry-pick this into 1.5 release branch as soon as possible.
Automatic merge from submit-queue (batch tested with PRs 36087, 39405)
federation-apiserver doesn't needs cluster-ip range
This option has been removed from genericserver options
and it is not being used by federation-apiserver anyways.
Fixes - #39388
Automatic merge from submit-queue (batch tested with PRs 38433, 36245)
Allow pods to define multiple environment variables from a whole ConfigMap
Allow environment variables to be populated from ConfigMaps
- ConfigMaps represent an entire set of EnvVars
- EnvVars can override ConfigMaps
fixes#26299
Automatic merge from submit-queue (batch tested with PRs 39075, 39350, 39353)
Move pkg/api.{Context,RequestContextMapper} into pkg/genericapiserver/api/request
**Based on #39350**
Automatic merge from submit-queue (batch tested with PRs 39250, 39206)
WIP: work around for IP and port Allocator repair race
Fixes#37488
WIP: This does the IP allocator but not the port allocator yet. Sending for review before I clone the deltas for ports.
Idea: force the repair loop to detect a leak 3 times in a row before actually releasing the IP. That should allow any distributed races to resolve. It's a little hacky, but without mutual exclusion or proper transactions, it works.
Automatic merge from submit-queue
Enable update tests in federated namespace controller
Previously it was disabled because of flakiness.
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
remove unneeded authenticator dependencies from genericapiserver
Refactors the authenticator options to remove unneeded dependencies.
@sttts
Automatic merge from submit-queue
Fix typo for federation/*
**What this PR does / why we need it**:
Increase code readability for this new member in v1.5
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:
Could we develop a typo-fix bot along with a k8s terminology dictionary ?
**Release note**:
```release-note
```
Automatic merge from submit-queue
register batch/jobs to federation-apiserver
register batch/jobs api objects to federation-apiserver
**Release note**:
```release-note
Federation: Add `batch/jobs` API objects to federation-apiserver
```
@quinton-hoole @nikhiljindal @deepak-vij
#34261
This enables testing when the dnsprovider is used externally (with a
mock Route53 API, as we do in kops), and also might be useful for
constructing with a particular client instance with extra options.
Automatic merge from submit-queue
fix typo on federation/pkg/dnsprovider/plugins.go
**What this PR does / why we need it**: Increase code readability
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: just fix one minor mistake as first contribution
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 37468, 36546, 38713, 38902, 38614)
Remove extensions/v1beta1 Job
Fixes https://github.com/kubernetes/kubernetes/issues/32763. This endpoint was deprecated in 1.5 and was planned to be removed in 1.6.
**Release note**:
```release-note
Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator.
```
Automatic merge from submit-queue
Rename root BUILD to BUILD.bazel, and rename build-tools/ back to build/
**What this PR does / why we need it**: undoes #35453. `build-tools/` was intended to be a temporary workaround until https://github.com/bazelbuild/bazel/issues/552 was fixed, which is has been, as of bazel 0.4.1.
**Which issue this PR fixes**: fixes#38126
FYI @thockin
since this is likely to be rebase hell, my list of actions:
```console
$ git mv BUILD BUILD.bazel
$ git commit -a
$ git mv build-tools/ build/
$ sift -l 'build-tools' | xargs sed -i -e 's:build-tools/:build/:g'
$ git checkout HEAD docs/
$ git commit -a
```
Automatic merge from submit-queue (batch tested with PRs 38154, 38502)
Rename "release_1_5" clientset to just "clientset"
We used to keep multiple releases in the main repo. Now that [client-go](https://github.com/kubernetes/client-go) does the versioning, there is no need to keep releases in the main repo. This PR renames the "release_1_5" clientset to just "clientset", clientset development will be done in this directory.
@kubernetes/sig-api-machinery @deads2k
```release-note
The main repository does not keep multiple releases of clientsets anymore. Please find previous releases at https://github.com/kubernetes/client-go
```
Automatic merge from submit-queue
genericapiserver: unify swagger and openapi in config
- make swagger config customizable
- remove superfluous `Config.Enable*` flags for OpenAPI and Swagger.
This is necessary for downstream projects to tweak the swagger spec.
Automatic merge from submit-queue
Register metav1.GetOptions with federation APIs.
PR #37693 missed a spot.
cc @kubernetes/sig-federation @nikhiljindal
Automatic merge from submit-queue (batch tested with PRs 38354, 38371)
Add GetOptions parameter to Get() calls in client library
Ref #37473
This PR is super mechanical - the non trivial commits are:
- Update client generator
- Register GetOptions in batch/v2alpha1 group
Automatic merge from submit-queue (batch tested with PRs 38278, 37770)
Refactor REST storage to use generic defaults
This removes the repetition in the REST storage builders by moving the logic to `restoptions.ApplyOptions`. `registry.StorageWithCacher`/`generic.StorageDecorator` no longer assume that they can build the `keyFunc` for arbitrary objects. `restoptions.ApplyOptions` uses the `registry.Store`'s `KeyFunc` for its call to `generic.StorageDecorator`.
```release-note
Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data.
```
Automatic merge from submit-queue
Add CoreDNS as Federation DNS provider
This PR contains following
1. DNS provider implementation for CoreDNS with etcd as backend
2. Script for optionally deploying CoreDNS in federation control plane
Still to do:
1. HA for CoreDNS service need to be handled, would be taken up in subsequent PR
2. Need to configure CoreDNS as fallback DNS to in-cluster KubeDNS(Manual Step). Need to consider automation.
Request for review: @quinton-hoole @madhusudancs @nikhiljindal @kshafiee @deepak-vij
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue (batch tested with PRs 36071, 32752, 37998, 38350, 38401)
Pass addressable values to DeepCopy
Extracted from https://github.com/kubernetes/kubernetes/pull/35728
These are the places we are currently calling DeepCopy incorrectly, and we need to fix, even if we don't pick up the changes to DeepCopy in #35728:
* creating a new cloner means we have no generated functions registered
* passing non-addressable values doesn't pick up generated deep copy functions, and forces us into reflective mode
Automatic merge from submit-queue
[Federation] Implement dry run support in kubefed init
This one implements one of the TODO items pending in the previous set of kubefed PRs.
This one is done on top of another todo PR https://github.com/kubernetes/kubernetes/pull/36310 which is being reviewed separately.
Please review only the last 2 commits in this one.
The design doc PR for kubefed is at https://github.com/kubernetes/kubernetes/pull/34484.
cc @kubernetes/sig-cluster-federation @madhusudancs @nikhiljindal
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
[Federation] `kubefed init` now supports dry run mode.
```
Automatic merge from submit-queue
[Federation] Make federation etcd PVC size configurable
This one implements one of the many TODO items pending in the previous set of kubefed PRs.
The design doc PR is at https://github.com/kubernetes/kubernetes/pull/34484
cc @kubernetes/sig-cluster-federation @madhusudancs
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
[Federation] kubefed init now has a new flag, --etcd-pv-capacity, which can be used to configure the persistent volume capacity for etcd.
```
Automatic merge from submit-queue (batch tested with PRs 37032, 38119, 38186, 38200, 38139)
Detect long-running requests from parsed request info
Follow up to https://github.com/kubernetes/kubernetes/pull/36064
Uses parsed request info to more tightly match verbs and subresources
Removes regex-based long-running request path matching (which is easily fooled)
```release-note
The --long-running-request-regexp flag to kube-apiserver is deprecated and will be removed in a future release. Long-running requests are now detected based on specific verbs (watch, proxy) or subresources (proxy, portforward, log, exec, attach).
```
Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)
remove unnecessary fields from genericapiserver config
Cleans up some unnecessary fields in the genericapiserver config.
This removes all dependencies on Config during cert generation, only operating
on ServerRunOptions. This way we get rid of the repeated call of Config.Complete
and cleanly stratify the GenericApiServer bootstrapping.
Automatic merge from submit-queue
Create GetOptions to allow passing "ResourceVersion" to GET requests.
Ref #37473
@kubernetes/sig-api-machinery @smarterclayton
Automatic merge from submit-queue
Enable updates in federated ingress controller tests
And make channel in test helper non-blocking.
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Reuse fields and labels
This should significantly reduce memory allocations in apiserver in large cluster.
Explanation:
- every kubelet is refreshing watch every 5-10 minutes (this generally is not causing relist - it just renews watch)
- that means, in 5000-node cluster, we are issuing ~10 watches per second
- since we don't have "watch heartbets", the watch is issued from previously received resourceVersion
- to make some assumption, let's assume pods are evenly spread across pods, and writes for them are evenly spread - that means, that a given kubelet is interested in 1 per 5000 pod changes
- with that assumption, each watch, has to process 2500 (on average) previous watch events
- for each of such even, we are currently computing fields.
This PR is fixing this problem.
Automatic merge from submit-queue
Remove ExportOptions from api/internal and use unversioned
Should only have one internal object in use
Part of #37530
Automatic merge from submit-queue
move parts of the mega generic run struct out
This splits the main `ServerRunOptions` into composeable pieces that are bindable separately and adds easy paths for composing servers to run delegating authentication and authorization.
@sttts @ncdc alright, I think this is as far as I need to go to make the composing servers reasonable to write. I'll try leaving it here
Automatic merge from submit-queue
Fix package aliases to follow golang convention
Some package aliases are not not align with golang convention https://blog.golang.org/package-names. This PR fixes them. Also adds a verify script and presubmit checks.
Fixes#35070.
cc/ @timstclair @Random-Liu
Automatic merge from submit-queue
Fix infinte loop in federated ingress controller
Previously ingress controller was constantly scheduling reconcilation, even if no updates were needed. That behavior creates a big mess in the logs and consumes resources.
This pr also fixes the stop function for federated ingress controller.
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Revision handling in federated deployment controller
Deployment controller in regular kubernetes automatically adds an annotation in deployment. This causes a bit of confusion in controller and tests. This PR skips revision annotation in checks. In the next K8S release we will need to have better support for deployment revisions.
Helps with #36588
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Stop deleting underlying services when federation service is deleted
Fixes https://github.com/kubernetes/kubernetes/issues/36799
Fixing federation service controller to not delete services from underlying clusters when federated service is deleted.
None of the federation controller should do this unless explicitly asked by the user using DeleteOptions. This is the only federation controller that does that.
cc @kubernetes/sig-cluster-federation @madhusudancs
```release-note
federation service controller: stop deleting services from underlying clusters when federated service is deleted.
```
Automatic merge from submit-queue
Fixing the logic to select first cluster in federated ingress controller
Ref https://github.com/kubernetes/kubernetes/issues/36074.
Before this change, ingress controller was using cluster with clusterIndex = 0 as the first cluster to create the ingress in.
But the ordering of clusters can change and hence ingress controller ended up creating the ingress in multiple clusters.
This PR fixes it by using an annotation on federated ingress. Controller now picks up a cluster randomly as the first cluster and creates ingress there. This clusters name is stored as an annotation on the federated ingress. Contoller does not create an ingress in any other cluster if this annotation is set on the federated ingress and IP has not been propagated. Once IP has been propagated, controller creates the ingress in all clusters.
cc @kubernetes/sig-cluster-federation @madhusudancs
Automatic merge from submit-queue
Fix doc links in Federation readme
**What this PR does / why we need it**:
The user guide and admin guide links were swapped round
**Release note**: NONE
Automatic merge from submit-queue
populate ready replicas and aviable replicas to federated replicaset …
populate ready replicas and aviable replicas to federated replicaset status
@nikhiljindal #33312#29481#32771
@deepak-vij
Automatic merge from submit-queue
Fixing the ingress annotation key in federated ingress controller
Fixing a few issues found during debugging https://github.com/kubernetes/kubernetes/issues/36074.
cc @kubernetes/sig-cluster-federation @madhusudancs @bprashanth
Automatic merge from submit-queue
[Federation][join-flags] Add flags for cluster context and secret names while joining clusters to federation.
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make
`kubefed join` consistent with `kubefed unjoin`. This also means
`--cluster-context` is now made a flag and defaults to cluster name if
unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Fixes: Issue #35954
cc @kubernetes/sig-cluster-federation @quinton-hoole @irfanurrehman
Automatic merge from submit-queue
Fields with omitempty tag should still be considered as optional
We've added an "+optional" tag while ago for optional fields. Before that OpenAPI spec generated assumed all fields with "omitempty" in their json tags are optional. This should be still the case (as well as +optional tag) until these two things happen:
- We update all documentation asking developers to use +optional (My bad, I should have added this after the +optional PR)
- We fix swagger 1.2 spec generator to use +optional instead of omitempty.
Fixes#37149
Automatic merge from submit-queue
Make desired objects in fed controller reconcilation function fully writable
When creating desired objects for sub-cluster in reconciliate functions we don't make full copies. This may result in unexpected race condition in the future when someone actually adds some object modifications (not needed in most cases but anyway - the cost of extra copy will be relatively small comparing to inter-cluster traffic). In case of ReplicaSet and Deployment the spec copy was a shallow one - ok for setting just replicas but also error-prone in the future.
cc: @quinton-hoole @nikhiljindal @madhusudancs
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make `kubefed
join` consistent with `kubefed unjoin`. This also means `--cluster-
context` is now made a flag and defaults to cluster name if unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Automatic merge from submit-queue
[Federation][Bugfix] Make the dns provider configurable in kubefed init
Fixes: Issue #36690
The design doc PR relating the whole feature for kubefed is at #34484.
cc @kubernetes/sig-cluster-federation @madhusudancs @nikhiljindal
Automatic merge from submit-queue
Federation test util fix - 1
* Make nil and empty map equivalent when testing ObjectMeta equivalence
* Add a function that waits for a particular content inside federated store
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Ensure proper serialization of updates and creates in federation test watcher
Fix for finalizer test problems. The issue there was that the updates were coming out of order. It was caused by firing updates in new goroutines in test watcher. The proper solution is to order them in a queue and fire in order on a single goroutine.
Ref: https://github.com/kubernetes/kubernetes/issues/36473#issuecomment-259605870
cc: @nikhiljindal @madhusudancs
Automatic merge from submit-queue
Adding cascading deletion support to federation replicaset and deployments
Forked from https://github.com/kubernetes/kubernetes/pull/36330
Ref #33612
Adding cascading deletion support for federated replicasets and deployments.
```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated replicasets and deployments. Setting it to false while deleting a federated replicaset or deployment also deletes the corresponding resource from all registered clusters.
```
Dr. Stefan Schimanski