Add test artifacts to the build. This lets you do:
tar -xzf kubernetes.tar.gz
tar -xzf kubernetes-test.tar.gz
cd kubernetes
go run ./hack/e2e.go -up -test -down
without having a git checkout.
by adding support for testing official release versions and writing a
simple script to first bring up and test the old version before
upgrading it and testing it again. Related to supporting in-place
upgrades of masters (#2524).
Make each test available for execution --times times. Acts like a
multi-deck shoe. (Otherwise this is easily scriptable outside the
e2e.go command).
Useful for "I want to walk away for a few hours leaving some end-to-ends
running", until we have more stress tests. Also useful for detecting
flakes.
Reporting is changed to break out Passed, Flaky and Failed. I chose to
keep all three lines even if --times isn't on, just for consistency in
scraping. Similarly, it always outputs the counts now. A report looks
like:
2014/12/09 07:31:21 Passed tests: goe2e.sh[100/100] guestbook.sh[100/100] private.sh[100/100] services.sh[100/100]
2014/12/09 07:31:21 Flaky tests: basic.sh[99/100] certs.sh[99/100] monitoring.sh[98/100] pd.sh[98/100] update.sh[98/100]
2014/12/09 07:31:21 Failed tests:
2014/12/09 07:31:21 8 test(s) failed.
Replaces the client public interface but leaves old references to "minions"
for a later refactor. Selects the path "nodes" for v1beta3 and "minions"
for older versions.
Currently, we run the e2e tests in whatever order readdir happens to
return, which is random on some filesystems, name sorted on others,
create order on others, etc. Eventually, we may want to be
automatically hermetic between e2e tests (especially as we introduce
more resource destructive tests), but until then, it would be useful
if we permute the test order randomly between runs to ensure that
developers don't accidentally rely on a particular order. This
introduces a form of forced hermeticism, since improper state cleanup
from one test may not perturb a given test, but there's probably *a*
test in the suite that the order will perturb, so the RNG will find
that order eventually.
Adds logging of the generated seed, and an --orderseed argument that
can be used to re-run in the same order. Also sorts the pass/fail list
now for easier human reading.
Minor usability nuisance: If you run:
go run hack/e2e.go -v -test
.. and you don't happen to have an up e2e cluster, it should fail
fast, rather than chugging through every test and having them fall
over.
This commit brings two main changes, notably:
Two new options that can be set as environment variables
- DOCKER_OPTS: any arbitrary set of docker options. Example: --tlsverify
- DOCKER_NATIVE: This forces the use of the native docker available.
This is most useful if you're on OS X and do not want
to use boot2docker.
Now uses 'docker cp' instead of tar piping to transfer files. This
currently must be done by copying the binaries off of the docker volume
and into a local filesystem (/tmp) before a docker cp is done. This
workaround will no longer be necessary after bug fix
https://github.com/docker/docker/pull/8509 makes it into stable.
This was necessary because the tar | tar method was creating corrupted
archives on OS X even with the < /dev/null workaround.
Brendan Burns