Automatic merge from submit-queue
Speed up dockerized builds
This PR speeds up dockerized builds. First, we make sure that we are as incremental as possible. The bigger change is that now we use rsync to move sources into the container and get data back out.
To do yet:
* [x] Add a random password to rsync. This is 128bit MD4, but it is better than nothing.
* [x] Lock down rsync to only come from the host.
* [x] Deal with remote docker engines -- this should be necessary for docker-machine on the mac.
* [x] Allow users to specify the port for the rsync daemon. Perhaps randomize this or let docker pick an ephemeral port and detect the port?
* [x] Copy back generated files so that users can check them in. This is done for `zz_generated.*` files generated by `make generated_files`
* [x] This should include generated proto files so that we can remove the hack-o-rama that is `hack/hack/update-*-dockerized.sh`
* [x] Start "versioning" the build container and the data container so that the CI system doesn't have to be manually kicked.
* [x] Get some benchmarks to qualify how much faster.
This replaces #28518 and is related to #30600.
cc @thockin @spxtr @david-mcmahon @MHBauer
Benchmarks by running `make clean ; sync ; time bash -xc 'time build/make-build-image.sh ; time sync ; time build/run.sh make ; time sync; time build/run.sh make'` on a GCE n1-standard-8 with PD-SSD.
| setup | build image | sync | first build | sync | second build | total |
|-------|-------------|----- |----------|------|--------------|------|
| baseline | 0m11.420s | 0m0.812s | 7m2.353s | 0m42.380s | 7m8.381s | 15m5.348s |
| this pr | 0m10.977s | 0m15.168s | 7m31.096s | 1m55.692s | 0m16.514s | 10m9.449s |
make-generated-{protobuf,runtime}.sh was doing some really nasty stuff with how
the build container was managed in order to copy results out. Since we have
more flexibility to grab results out of the build container, we can now avoid
all of this. Ideally we wouldn't have `hack` calling `build` at all, but we
aren't there yet.
Automatic merge from submit-queue
Autoconvert cleanups
A few accumulated cleanups to conversion, and then a big change. Please review commit-by-commit. Verified by nuking all generated files and comparing the new results.
cc @lavalamp
Previously we refused to emit 'autoConvert_*' functions if any field was not
convertible. The way around this was to write manual Conversion functions, but
to do so safely you must handle every fields. Huge opportunity for errors.
This PR cleans up the filtering such that it only operates on types that should
be converted (remove a lot of code) and tracks when fields are skipped. In
that case, it emits an 'autoConvert' function but not a public 'Convert'
function. If there is no manual function, the compile will fail.
This also means that manual conversion functions can call autoConvert functions
and then "patch up" what they need.
rename plugin/pkg/client/auth/plugins.go package name to auth
add the plugin import line in client-gen
update import_known_versions for release_1_5 clientset
change copy.sh
Automatic merge from submit-queue
Switch ScheduledJob controller to use clientset
**What this PR does / why we need it**:
This is part of #25442. I've applied here the same fix I've applied in the manual client in #29187, see the 1st commit for that (@caesarxuchao we've talked about it in #29856).
@deads2k as promised
@janetkuo ptal
Automatic merge from submit-queue
Improvements on OpenAPI spec generation
- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation
Reference: #13414
**Release note**:
```release-note
Generate separate OpenAPI spec for each API GroupVersion on /<Group>/<Version>/swagger.json
```
- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation
Automatic merge from submit-queue
Avoid sorting lists when unnecessary
I've seen ThreadSafeMap::List consuming ~30% of whole CPU usage, spending the whole time in sorting (while it is in fact completely unneded).
Kubernetes Submit Queue