- remove skip list from conformance-test.sh and filter by the new tag
- remove experimental api tests from conformance test suite
- remove all tests from conformance test suite which are either
restricted to e.g. gce, gke, aws or require SSH
As discussed @gmarek the given test does not belong into the conformance test
suite because it makes a lot of static assumptions about the cgroup setup of the
nodes which cannot be fulfilled by all cluster providers. Depending on the
installation the kubelet is not allowed to move around process
into specific containers.
Fixes https://github.com/mesosphere/kubernetes-mesos/issues/439.
Fix formatting and quote variable.
Fix comments from a previous review.
Move instructions to the top.
Change from using tag to using branch.
Improve documentation of conformance test.
Use stable random seed.
Fix bug where there was a double quote as a prefix and suffix to the
skip regexp.
Remove .* not needed in regex.
Skip a few more tests that are not reliable for me in the
Conformance test environment.
Use KUBE_CONFIG_FILE instead of AUTH_CONFIG (and CERT_DIR).
Pass GINKGO_TEST_ARGS for a subset of e2e tests which
@erictune has deemed initially sufficient for conformance.
Allow GINKGO_TEST_ARGS to pass through hack/ginkgo-e2e.sh.
Set NUM_MINIONS (need better way to handle this).
Remove use of "KUBERNETES_CONFORMANCE_TEST" variable
and use of KUBERNETES_PROVIDER="conformance_test" convention,
both of which have no apparent purpose.
Allow unset testContext.provider in test/e2e/e2e_test.go
Allow testContext.Provider to be unset in providerIs().
A conformance test is a test you run against a cluster that is already
set up. We would use it to test a hosted kubernetes service to make
sure that it meets a bar for quality. Also, a getting-started-guide
author, who has not implemented a complete set of cluster/...
scripts (that is, the getting-started-guide has some non-automated steps)
can use this to see which e2e tests pass on a cluster.
To be done in future PRs:
- disable tests which can't possibly run in a conformance test
because they require things like cluster ssh.
- document that when we accept a getting-started-guide, that
people should run the conformance test against their cluster
(unless they already have cluster/... scripts.
I ran this against a GCE cluster and 22 tests passed.
Karl Isenberg