This commit deletes cmd/e2e and updates hack/ginkgo-e2e.sh to use the
'ginkgo' command instead. All logic from cmd/e2e/e2e.go and
test/e2e/driver.go have been combined into the new file
test/e2e/e2e_test.go.
The test tarball now includes a built version of the test/e2e test
binary, which includes all tests under test/e2e. This was accomplished
by updating the build scripts to use 'go test -c' when a target name
ended with '.test', and adding a dependency on test/e2e/e2e.test.
This prebuilt test binary is passed to the Ginkgo runner in
hack/ginkgo-e2e.sh. In a future change, we can add support to run
Ginkgo against the source tree if it is available.
This change is generally intended to have no externally visible changes,
aside from the following caveats:
- The -t/--tests flag has been removed
- Calling cmd/e2e/e2e directly obviously won't work, but that was never
intended to be supported anyway
- If the GINKGO_PARALLEL environment variable is set to y, then ginkgo
will run test specs in parallel. (Currently defaults to n, since some
tests are broken in this mode.)
Additionally, several tests which made poor assumptions about cwd or
used testContext before it had been set have been fixed.
This commit deletes cmd/e2e and updates hack/ginkgo-e2e.sh to use the
'ginkgo' command instead. All logic from cmd/e2e/e2e.go and
test/e2e/driver.go have been combined into the new file
test/e2e/e2e_test.go.
Additionally, several tests which made poor assumptions about cwd or
used testContext before it was set have been fixed.
This change is generally intended to have no externally visible changes,
aside from the following caveats:
- The -t/--tests flag has been removed
- Calling cmd/e2e/e2e directly obviously won't work, but that was never
supported anyway
- If the GINKGO_PARALLEL environment variable is set to y, then ginkgo
will run test specs in parallel. (Currently defaults to n, since some
tests are broken in this mode.)
We were specifying a region, but naming it as a zone in util.sh
The zone matters just as much as the region, e.g. for EBS volumes.
We also change the config to require a Zone, not a Region.
But we fallback to get the information from the metadata service.
A conformance test is a test you run against a cluster that is already
set up. We would use it to test a hosted kubernetes service to make
sure that it meets a bar for quality. Also, a getting-started-guide
author, who has not implemented a complete set of cluster/...
scripts (that is, the getting-started-guide has some non-automated steps)
can use this to see which e2e tests pass on a cluster.
To be done in future PRs:
- disable tests which can't possibly run in a conformance test
because they require things like cluster ssh.
- document that when we accept a getting-started-guide, that
people should run the conformance test against their cluster
(unless they already have cluster/... scripts.
I ran this against a GCE cluster and 22 tests passed.
e2e auth breakage it caused. The fix is to not set project/zone/kube_master
to the empty string partway through the script, which I should have
realized was a bad idea in the first place.
Use ginkgo's native support for JUnit in order to generate the XML file.
This is a first step in better integration of our e2e tests with
Jenkins. In order to improve the logged information, we will probably
need to have more native ginkgo tests but this step allows us to see
what Jenkins can already do with this information and what we need to
tweak to improve it.
Tested by running the full e2e tests and inspecting the contents of
junit.xml on the top of the tree.
Textual output is still generated on the console to keep the current
goe2e.sh logs available until the full conversion of our Jenkins
instance to use the JUnit XML is completed.
* Add --orderseed, shuffle order every time, report order for repeatability
* Add --times, acts like a multi-deck shoe
* Remove fixed numbering in TAP output (this is actually not needed;
TAP output is just done by outputting what assertion count you're on.)
This is essentially just a port of f3a992aa and 369064c6 (minus
reporting, which can be handled later when we make TAP, etc, better).
This syntax is akin to what Python unittest uses for running a subset of the tests.
If a test gets skipped, log it. If an invalid test test is passed to --test, warn about it.
Add test artifacts to the build. This lets you do:
tar -xzf kubernetes.tar.gz
tar -xzf kubernetes-test.tar.gz
cd kubernetes
go run ./hack/e2e.go -up -test -down
without having a git checkout.
Jeff Grafton