41 Commits (CC-5545/upgrade-hds-packages)

Author	SHA1	Message	Date
R.B. Boyer	eb06db0c69	sdk: update testutil.WaitForLeader to not use the v1 catalog api (#19146)	1 year ago
R.B. Boyer	df930a59ac	chore: fix ce/ent drift in sdk and testing/deployer submodules (#19041)	1 year ago
James Hartig	b2e21c103f	consul operator raft transfer-leader should send the id (#17107) ... Fixes #16955 Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	1 year ago
Dan Stough	a1cd3f8329	feat: add experiments flag to testserver sdk (#18541)	1 year ago
Vijay	2f20c77e4d	Displays Consul version of each nodes in UI nodes section (#17754) ... * update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> --------- Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>	1 year ago
Ronald	17f4689379	backport ent changes to oss (#17614) ... * backport ent changes to oss * Update .changelog/_5669.txt Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> --------- Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>	1 year ago
cskh	82915d225f	Test: add noCleanup to TestServer stop (#16919)	2 years ago
Ronald	b64674623e	Copyright headers for missing files/folders (#16708) ... * copyright headers for agent folder	2 years ago
Eric Haberkorn	57e2493415	allow setting locality on services and nodes (#16581)	2 years ago
Derek Menteer	f52f3c5afc	Fix SDK to support older versions of Consul. (#15423) ... This change was necessary, because the configuration was always generated with a gRPC TLS port, which did not exist in Consul 1.13, and would result in the server failing to launch with an error. This code checks the version of Consul and conditionally adds the gRPC TLS port, only if the version number is greater than 1.14.	2 years ago
Kyle Schochenmaier	bf0f61a878	removes ioutil usage everywhere which was deprecated in go1.16 (#15297) ... * update go version to 1.18 for api and sdk, go mod tidy * removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward. Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	2 years ago
freddygv	fac3ddc857	Use internal server certificate for peering TLS ... A previous commit introduced an internally-managed server certificate to use for peering-related purposes. Now the peering token has been updated to match that behavior: - The server name matches the structure of the server cert - The CA PEMs correspond to the Connect CA Note that if Conect is disabled, and by extension the Connect CA, we fall back to the previous behavior of returning the manually configured certs and local server SNI. Several tests were updated to use the gRPC TLS port since they enable Connect by default. This means that the peering token will embed the Connect CA, and the dialer will expect a TLS listener.	2 years ago
Chris S. Kim	b374250d08	Add debugging logs to freeport	2 years ago
Luke Kysow	988e1fd35d	peering: default to false (#13963) ... * defaulting to false because peering will be released as beta * Ignore peering disabled error in bundles cachetype Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: freddygv <freddy@hashicorp.com> Co-authored-by: Matt Keeler <mjkeeler7@gmail.com>	2 years ago
R.B. Boyer	af04851637	peering: move peer replication to the external gRPC port (#13698) ... Peer replication is intended to be between separate Consul installs and effectively should be considered "external". This PR moves the peer stream replication bidirectional RPC endpoint to the external gRPC server and ensures that things continue to function.	2 years ago
Ashwin Venkatesh	7568f3a102	Add support for 'Partition' and 'RetryJoin' (#12126) ... - Adding a 'Partition' and 'RetryJoin' command allows test cases where one would like to spin up a Consul Agent in a non-default partition to test use-cases that are common when enabling Admin Partition on Kubernetes.	3 years ago
Dan Upton	ca3aca92c4	[OSS] Remove remaining references to master (#11827)	3 years ago
Daniel Nephin	5a61893642	testing: use httptest with freeport	3 years ago
FFMMM	4ddf973a31	add root_cert_ttl option for consul connect, vault ca providers (#11428) ... * add root_cert_ttl option for consul connect, vault ca providers Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Chris S. Kim <ckim@hashicorp.com> * add changelog, pr feedback Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> * Update .changelog/11428.txt, more docs Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> * Update website/content/docs/agent/options.mdx Co-authored-by: Kyle Havlovitz <kylehav@gmail.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>	3 years ago
Sergey Matyukevich	06f3ccebce	Allow configuring graceful stop in testutil (#10566) ... * Allow configuring graceful stop in testutil Signed-off-by: Sergey Matyukevich <s.matyukevich@gmail.com> * add a changelog Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	3 years ago
David Bariod	797bfb7b51	remove a race condition in sdk testutil server stop (#10342) ... * remove a race condition in sdk testutil server stop * ensure the process has actually stopped	4 years ago
John Eikenberry	1266bfd65b	[SDK] change all cases of *testing.T to testing.TB ... Using the interface opens up the use of all methods to benchmarks as well as tests.	4 years ago
R.B. Boyer	a2a8e9c783	connect: intentions are now managed as a new config entry kind "service-intentions" (#8834) ... - Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older copies of consul will not see new config entries it doesn't understand replicate down. - Add shim conversion code so that the old API/CLI method of interacting with intentions will continue to work so long as none of these are edited via config entry endpoints. Almost all of the read-only APIs will continue to function indefinitely. - Add new APIs that operate on individual intentions without IDs so that the UI doesn't need to implement CAS operations. - Add a new serf feature flag indicating support for intentions-as-config-entries. - The old line-item intentions way of interacting with the state store will transparently flip between the legacy memdb table and the config entry representations so that readers will never see a hiccup during migration where the results are incomplete. It uses a piece of system metadata to control the flip. - The primary datacenter will begin migrating intentions into config entries on startup once all servers in the datacenter are on a version of Consul with the intentions-as-config-entries feature flag. When it is complete the old state store representations will be cleared. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up. - The secondary datacenters continue to run the old intentions replicator until all servers in the secondary DC and primary DC support intentions-as-config-entries (via serf flag). Once this condition it met the old intentions replicator ceases. - The secondary datacenters replicate the new config entries as they are migrated in the primary. When they detect that the primary has zeroed it's old state store table it waits until all config entries up to that point are replicated and then zeroes its own copy of the old state store table. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up.	4 years ago
Paul Burlumi	bb2b5dd871	testutil: use TestingTB instead of testing.TB	4 years ago
Daniel Nephin	fe2f80c3a1	Use SIGABRT to get a stack trace when the timeout is hit	4 years ago
Daniel Nephin	8e67d8eaeb	sdk: mitigate api test timeout ... Occasionally we are seeing the go-test-api job timeout at 10 minutes. Looking at the stack trace I saw the following: 1. Lots of tests blocked on server.Stop in NewTestServerConfigT. This suggests that SIGINT is being sent to the server, but the server is not properly shutting down. 2. Over 20k goroutines that look like this: goroutine 16355 [select, 8 minutes]: net/http.(*persistConn).readLoop(0xc004270240) /usr/local/go/src/net/http/transport.go:2099 +0x99e created by net/http.(*Transport).dialConn /usr/local/go/src/net/http/transport.go:1647 +0xc56 Issue 1 seems to be the main problem, but debugging that directly is not possible because our buffered logs do not get sent when the tests timeout. To mitigate this problem I've added a timeout to the cmd.Wait() to force kill the process and return an error. Unfortunately because we retry this operation, we still may not see the cause because the next attempt will likely pass. I'm tempted to remove the retry around NewTestServerConfigT. Issue 2 seems to be caused by not closing the response body. Since the request is performed many times in a loop, many goroutines are created and are not closed until the response body is closed.	4 years ago
Daniel Nephin	51efba2c7d	testutil: NewLogBuffer - buffer logs until a test fails ... Replaces #7559 Running tests in parallel, with background goroutines, results in test output not being associated with the correct test. `go test` does not make any guarantees about output from goroutines being attributed to the correct test case. Attaching log output from background goroutines also cause data races. If the goroutine outlives the test, it will race with the test being marked done. Previously this was noticed as a panic when logging, but with the race detector enabled it is shown as a data race. The previous solution did not address the problem of correct test attribution because test output could still be hidden when it was associated with a test that did not fail. You would have to look at all of the log output to find the relevant lines. It also made debugging test failures more difficult because each log line was very long. This commit attempts a new approach. Instead of printing all the logs, only print when a test fails. This should work well when there are a small number of failures, but may not work well when there are many test failures at the same time. In those cases the failures are unlikely a result of a specific test, and the log output is likely less useful. All of the logs are printed from the test goroutine, so they should be associated with the correct test. Also removes some test helpers that were not used, or only had a single caller. Packages which expose many functions with similar names can be difficult to use correctly. Related: https://github.com/golang/go/issues/38458 (may be fixed in go1.15) https://github.com/golang/go/issues/38382#issuecomment-612940030	4 years ago
Iryna Shustava	03366573d7	sdk: Use /v1/status/leader endpoint when starting a test server (#8192) ... Switch from /v1/agent/self to /v1/status/leader when checking if the test server has come up successfully in the waitForAPI function. Previously, the test server was relying (probably not intentionally) on the default value of the acl_enforce_version_8 in the TestConfig, which was false. So if you create a test server and enabled ACLs, they would not be enforced and the server would be able to come up pretty quickly because /v1/agent/self would return a 200 status pretty much as soon as the agent is running and most likely before leader election is finished. Now that we have removed acl_enforce_version_8 property (equivalent to being true by default) if you've created a test server with ACLs enabled, it will need to wait for leader election and for ACLs to be initialized before it'll get a successful response from the /v1/agent/self. Note: With this change, waitForAPI function no longer requires a 200 response status from the v1/status/leader endpoint. This is because in some tests, namely TestAPI_AgentLeave, we are only running clients, and this endpoint returns a 500 status.	4 years ago
R.B. Boyer	ffb9c7d6f7	acl: remove the deprecated `acl_enforce_version_8` option (#7991) ... Fixes #7292	5 years ago
Matt Keeler	b57c2b78fd	Unflake the TestAPI_AgentConnectCALeaf test (#7142) ... * Unflake the TestAPI_AgentConnectCALeaf test * Modify the WaitForActiveCARoot to actually verify that at least one root exists Also verify that the active root id field is set	5 years ago
Chris Piraino	336f62fa99	Fix up formatting in sdk package (#7109)	5 years ago
Song Yihan	04305be555	tests: fix zombie consul process while invoking TestServer.Stop() method in sdk/testutil in Windows (#6032) ... * Fix zombie consul process in Windows Windows doesn't support Interrupt signal, thus while stop it on Windows platform it would fail and left zombie consul process	5 years ago
Paul Banks	1807af552e	Fix TestAPI_DiscoveryChain_Get flake (#7082)	5 years ago
Mike Morris	1112c696cf	sdk: add NewTestServerT, deprecate NewTestServer (#6761) ... * prevent nil pointer deref from t.Logf usage * enforce non-nil *testing.T in NewTestServerT	5 years ago
Matt Keeler	b9fe39d5aa	Nil checks in the testWriter to prevent using a bad testing.TB (#6517) ... Also needed to update some funcs that were taking a *testing.T to use a testing.TB. This prevents passing a nil pointer as a non-nil interface value and thus making it impossible to detect nil before using the interfaces functions.	5 years ago
Matt Keeler	6d4930da93	Dont crash in the testutil server creation (#6516)	5 years ago
R.B. Boyer	f74244dde8	api/watch: reduce timing dependence on tests of watch behavior ... Also for debugging purposes send the stdout/stderr streams from consul processes spawned for API tests to testing.T.Logf	5 years ago
R.B. Boyer	f9496dc627	sdk: add freelist tracking and ephemeral port range skipping to freeport ... This should cut down on test flakiness. Problems handled: - If you had enough parallel test cases running, the former circular approach to handling the port block could hand out the same port to multiple cases before they each had a chance to bind them, leading to one of the two tests to fail. - The freeport library would allocate out of the ephemeral port range. This has been corrected for Linux (which should cover CI). - The library now waits until a formerly-in-use port is verified to be free before putting it back into circulation.	5 years ago
Mike Morris	65be58703c	connect: remove managed proxies (#6220) ... * connect: remove managed proxies implementation and all supporting config options and structs * connect: remove deprecated ProxyDestination * command: remove CONNECT_PROXY_TOKEN env var * agent: remove entire proxyprocess proxy manager * test: remove all managed proxy tests * test: remove irrelevant managed proxy note from TestService_ServerTLSConfig * test: update ContentHash to reflect managed proxy removal * test: remove deprecated ProxyDestination test * telemetry: remove managed proxy note * http: remove /v1/agent/connect/proxy endpoint * ci: remove deprecated test exclusion * website: update managed proxies deprecation page to note removal * website: remove managed proxy configuration API docs * website: remove managed proxy note from built-in proxy config * website: add note on removing proxy subdirectory of data_dir	5 years ago
Freddy	0b9111714b	Update TestServer creation in sdk/testutil (#6084) ... * Retry the creation of the test server three times. * Reduce the retry timeout for the API wait to 2 seconds, opting to fail faster and start over. * Remove wait for leader from server creation. This wait can be added on a test by test basis now that the function is being exported. * Remove wait for anti-entropy sync. This is built into the existing WaitForSerfCheck func, so that can be used if the anti-entropy wait is needed	5 years ago
Jeff Mitchell	4243c3ae42	Move internal/ to sdk/ (#5568) ... * Move internal/ to sdk/ * Add a readme to the SDK folder	6 years ago