This gets rid of bashWrap() and of running bash -s passing it a script
on stdin. Also get rid of most of the runBash*() helpers.
In particular, `go run hack/e2e.go -build` will now work when it needs
to ask whether it is OK to download a large docker image, it used to
fail since stdin was not available for the bash script using the `read`
command.
Tested by running a full `hack/e2e-test.sh` run including the build
stage without the docker image loaded, which used to fail before this
change.
* Add a test/e2e/shell.go that slurps up everything in hack/e2e-suite
and runs it as a bash test, borrowing all the code from hack/e2e.go.
* Rip out all the crap in hack/e2e.go that deal with multiple tests
* Move hack/e2e-suite/goe2e.sh to hack/ginkgo-e2e.sh so that it
doesn't get slurped up.
Simply incorporate some of the boilerplate from hack/e2e.go into the
scripts in hack/e2e-suite.
Use environment variables with default values to allow overrides in
kubectl command line and to use a versioned package root.
Tested:
- Ran `go run hack/e2e.go -test` on a test cluster.
- Ran the test cases individually.
- Ran hack/e2e-suite/goe2e.sh -t Pods to confirm it takes arguments.
- Also fixed cluster/test-network.sh (which should be more and more irrelevant.)
There are quite a few 'composite literal uses unkeyed fields' errors that I have kept out of this patch.
And there's a couple where vet just seems confused. These are the easiest ones.
by adding support for testing official release versions and writing a
simple script to first bring up and test the old version before
upgrading it and testing it again. Related to supporting in-place
upgrades of masters (#2524).
Make each test available for execution --times times. Acts like a
multi-deck shoe. (Otherwise this is easily scriptable outside the
e2e.go command).
Useful for "I want to walk away for a few hours leaving some end-to-ends
running", until we have more stress tests. Also useful for detecting
flakes.
Reporting is changed to break out Passed, Flaky and Failed. I chose to
keep all three lines even if --times isn't on, just for consistency in
scraping. Similarly, it always outputs the counts now. A report looks
like:
2014/12/09 07:31:21 Passed tests: goe2e.sh[100/100] guestbook.sh[100/100] private.sh[100/100] services.sh[100/100]
2014/12/09 07:31:21 Flaky tests: basic.sh[99/100] certs.sh[99/100] monitoring.sh[98/100] pd.sh[98/100] update.sh[98/100]
2014/12/09 07:31:21 Failed tests:
2014/12/09 07:31:21 8 test(s) failed.
Currently, we run the e2e tests in whatever order readdir happens to
return, which is random on some filesystems, name sorted on others,
create order on others, etc. Eventually, we may want to be
automatically hermetic between e2e tests (especially as we introduce
more resource destructive tests), but until then, it would be useful
if we permute the test order randomly between runs to ensure that
developers don't accidentally rely on a particular order. This
introduces a form of forced hermeticism, since improper state cleanup
from one test may not perturb a given test, but there's probably *a*
test in the suite that the order will perturb, so the RNG will find
that order eventually.
Adds logging of the generated seed, and an --orderseed argument that
can be used to re-run in the same order. Also sorts the pass/fail list
now for easier human reading.
Minor usability nuisance: If you run:
go run hack/e2e.go -v -test
.. and you don't happen to have an up e2e cluster, it should fail
fast, rather than chugging through every test and having them fall
over.
* Rewrite a bunch of the hack/ directory with modular reusable bash libraries.
* Have 'build/*' build on 'hack/*'. The stuff in build now just runs hack/* in a docker container.
* Use a docker data container to enable faster incremental builds.
* Standardize output to _output/{local,dockerized}/bin/OS/ARCH/*. This regularized placement makes cross compilation work.
* Move travis specific scripts under hack/travis
With new dockerized incremental builds, I can do a no-op `make quick-release` in ~30s. This is a significant improvement.
Zach Loafman