293 Commits (53cdda8d17dcdebad7385e2c0c521822fe522f3b)

Author	SHA1	Message	Date
Nitya Dhanushkodi	8dab825c36	troubleshoot: fixes and updated messages (#16294)	2 years ago
Thomas Eckert	2460ac99c9	API Gateway Envoy Golden Listener Tests (#16221) ... * Simple API Gateway e2e test for tcp routes * Drop DNSSans since we don't front the Gateway with a leaf cert * WIP listener tests for api-gateway * Return early if no routes * Add back in leaf cert to testing * Fix merge conflicts * Re-add kind to setup * Fix iteration over listener upstreams * New tcp listener test * Add tests for API Gateway with TCP and HTTP routes * Move zero-route check back * Drop generateIngressDNSSANs * Check for chains not routes --------- Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
cskh	fd61605ffb	upgrade test: fix flaky peering through mesh gateway (#16271)	2 years ago
malizz	247211de6a	add integration tests for troubleshoot (#16223) ... * draft * expose internal admin port and add proxy test * update tests * move comment * add failure case, fix lint issues * cleanup * handle error * revert changes to service interface * address review comments * fix merge conflict * merge the tests so cluster is created once * fix other test	2 years ago
cskh	5b7f36c2ce	integ test: fix retry upstream test (#16246)	2 years ago
Andrew Stucki	9bb0ecfc18	[API Gateway] Add integration test for HTTP routes (#16236) ... * [API Gateway] Add integration test for conflicted TCP listeners * [API Gateway] Update simple test to leverage intentions and multiple listeners * Fix broken unit test * [API Gateway] Add integration test for HTTP routes	2 years ago
cskh	ab5dac3414	upgrade test: peering with http router config entry (#16231) ... * upgrade test: peering with http router config entry	2 years ago
Andrew Stucki	8ff2974dbe	[API Gateway] Update simple test to leverage intentions and multiple listeners (#16228) ... * [API Gateway] Add integration test for conflicted TCP listeners * [API Gateway] Update simple test to leverage intentions and multiple listeners * Fix broken unit test * PR suggestions	2 years ago
Andrew Stucki	318ba215ab	[API Gateway] Add integration test for conflicted TCP listeners (#16225)	2 years ago
Derek Menteer	4f2ce60654	Fix peering acceptors in secondary datacenters. (#16230) ... Prior to this commit, secondary datacenters could not be initialized as peering acceptors if ACLs were enabled. This is due to the fact that internal server-to-server API calls would fail because the management token was not generated. This PR makes it so that both primary and secondary datacenters generate their own management token whenever a leader is elected in their respective clusters.	2 years ago
Andrew Stucki	3b9c569561	Simple API Gateway e2e test for tcp routes (#16222) ... * Simple API Gateway e2e test for tcp routes * Drop DNSSans since we don't front the Gateway with a leaf cert	2 years ago
Andrew Stucki	f4210d47dd	Add basic smoke test to make sure an APIGateway runs (#16217)	2 years ago
Anita Akaeze	d72ad5fb95	Merge pull request #4216 from hashicorp/NET-2252-add-assert-fortioname (#16212) ... NET-2252: integration tests: add assert.FortioName	2 years ago
cskh	e91bc9c058	feat: envoy extension - http local rate limit (#16196) ... - http local rate limit - Apply rate limit only to local_app - unit test and integ test	2 years ago
cskh	b4151780d6	Upgrade test: verify the agent token is working after upgrade (#16164) ... 1. Upgraded agent can inherit the persisted token and join the cluster 2. Agent token prior to upgrade is still valid after upgrade 3. Enable ACL in the agent configuration	2 years ago
wangxinyi7	906ebb97f6	change log level (#16128)	2 years ago
Anita Akaeze	f58d5bb221	NET-2087: Restart proxy sidecar during cluster upgrade (#16140)	2 years ago
Anita Akaeze	fcf18222c5	add assertions (#16087)	2 years ago
Dan Upton	e40b731a52	rate: add prometheus definitions, docs, and clearer names (#15945)	2 years ago
Anita Akaeze	08a19e532d	NO_JIRA: Add function to get container status before making api call (#16116)	2 years ago
cskh	f6da81c9d0	improvement: prevent filter being added twice from any enovy extension (#16112) ... * improvement: prevent filter being added twice from any enovy extension * break if error != nil * update test	2 years ago
cskh	9da61c1cec	Upgrade test: retain sidecar containers during upgrade. (#16100)	2 years ago
cskh	ffb81782de	Upgrade test: peering control plane traffic through mesh gateway (#16091)	2 years ago
cskh	5fa9ab28dc	integ test: remove hardcoded upstream local bind port and max number of envoy sidecar (#16092)	2 years ago
cskh	ebdb8e5fb2	flaky test: use retry long to wait for config entry upgrade (#16068) ... * flaky test: use retry long to wait for config entry upgrade * increase wait for rbac policy	2 years ago
cskh	dbaab52786	Post upgrade test validation: envoy endpoint and register service (#16067)	2 years ago
Dan Stough	bb71d045e1	test: run integration tests in parallel (#16035)	2 years ago
R.B. Boyer	96389eb266	test: container tests wait for available networks (#16045)	2 years ago
Dan Stough	91d6a81c14	test(integration): add access logging test (#16008)	2 years ago
John Murret	794277371f	Integration test for server rate limiting (#15960) ... * rate limit test * Have tests for the 3 modes * added assertions for logs and metrics * add comments to test sections * add check for rate limit exceeded text in log assertion section. * fix linting error * updating test to use KV get and put. move log assertion tolast. * Adding logging for blocking messages in enforcing mode. refactoring tests. * modified test description * formatting * Apply suggestions from code review Co-authored-by: Dan Upton <daniel@floppy.co> * Update test/integration/consul-container/test/ratelimit/ratelimit_test.go Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * expand log checking so that it ensures both logs are they when they are supposed to be and not there when they are not expected to be. * add retry on test * Warn once when rate limit exceed regardless of enforcing vs permissive. * Update test/integration/consul-container/test/ratelimit/ratelimit_test.go Co-authored-by: Dan Upton <daniel@floppy.co> Co-authored-by: Dan Upton <daniel@floppy.co> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
Anita Akaeze	4e154144a6	NET-2038: Add envoy assertion function of listener verification (#15969)	2 years ago
Dan Stough	e8dde59bd0	chore(ci): fix compat ent compat tests for sidecars and gateways (#15997)	2 years ago
R.B. Boyer	d59efd390c	test: general cleanup and fixes for the container integration test suite (#15959) ... - remove dep on consul main module - use 'consul tls' subcommands instead of tlsutil - use direct json config construction instead of agent/config structs - merge libcluster and libagent packages together - more widely use BuildContext - get the OSS/ENT runner stuff working properly - reduce some flakiness - fix some correctness related to http/https API	2 years ago
Dan Stough	6d2880e894	feat: add access logs to dataplane bootstrap rpc (#15951)	2 years ago
Matt Keeler	5afd4657ec	Protobuf Modernization (#15949) ... * Protobuf Modernization Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf Marshallers (protobuf and json) needed some changes to account for different APIs. Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package. This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces. * Fix go-mod-tidy make target to work on all modules	2 years ago
Eric Haberkorn	8d923c1789	Add the Lua Envoy extension (#15906)	2 years ago
cskh	1113314641	Refactoring the peering integ test to accommodate coming changes of o… (#15885) ... * Refactoring the peering integ test to accommodate coming changes of other upgrade scenarios. - Add a utils package under test that contains methods to set up various test scenarios. - Deduplication: have a single CreatingPeeringClusterAndSetup replace CreatingAcceptingClusterAndSetup and CreateDialingClusterAndSetup. - Separate peering cluster creation and server registration. * Apply suggestions from code review Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	2 years ago
Dan Upton	d4c435856b	grpc: `protoc` plugin for generating gRPC rate limit specifications (#15564) ... Adds automation for generating the map of `gRPC Method Name → Rate Limit Type` used by the middleware introduced in #15550, and will ensure we don't forget to add new endpoints. Engineers must annotate their RPCs in the proto file like so: ``` rpc Foo(FooRequest) returns (FooResponse) { option (consul.internal.ratelimit.spec) = { operation_type: READ, }; } ``` When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will be installed that writes rate-limit specs as a JSON array to a file called `.ratelimit.tmp` (one per protobuf package/directory). After running Buf, `make proto` will execute a post-process script that will ingest all of the `.ratelimit.tmp` files and generate a Go file containing the mappings in the `agent/grpc-middleware` package. In the enterprise repository, it will write an additional file with the enterprise-only endpoints. If an engineer forgets to add the annotation to a new RPC, the plugin will return an error like so: ``` RPC Foo is missing rate-limit specification, fix it with: import "proto-public/annotations/ratelimit/ratelimit.proto"; service Bar { rpc Foo(...) returns (...) { option (hashicorp.consul.internal.ratelimit.spec) = { operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT, }; } } ``` In the future, this annotation can be extended to support rate-limit category (e.g. KV vs Catalog) and to determine the retry policy.	2 years ago
Nitya Dhanushkodi	d382ca0aec	extensions: refactor serverless plugin to use extensions from config entry fields (#15817) ... docs: update config entry docs and the Lambda manual registration docs Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Eric <eric@haberkorn.co>	2 years ago
cskh	692a6edd7d	Upgrade test: test peering upgrade from an old version of consul (#15768) ... * upgrade test: test peering upgrade from an old version of consul NET-1809	2 years ago
Semir Patel	bafa5c7156	Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700)	2 years ago
cskh	04bf24c8c1	feat(ingress-gateway): support outlier detection of upstream service for ingress gateway (#15614) ... * feat(ingress-gateway): support outlier detection of upstream service for ingress gateway * changelog Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>	2 years ago
Dan Stough	98ef5f28dd	[OSS] security: update x/net module (#15737) ... Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
Eric Haberkorn	4268c1c25c	Remove the `connect.enable_serverless_plugin` agent configuration option (#15710)	2 years ago
cskh	3df68751f5	Flakiness test: case-cfg-splitter-peering-ingress-gateways (#15707) ... * integ-test: fix flaky test - case-cfg-splitter-peering-ingress-gateways * add retry peering to all peering cases Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	2 years ago
cskh	36f05bc8fb	integ-test: test consul upgrade from the snapshot of a running cluster (#15595) ... * integ-test: test consul upgrade from the snapshot of a running cluster * use Target version as default Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	2 years ago
Dan Stough	227fd14287	chore: updates from 1.14.2 release (#15633) ... * chore: updates from 1.14.2 release	2 years ago
Derek Menteer	8079686bf0	Add 1.14.1 release updates. (#15514) ... Add post-release changes for 1.14.1 updates.	2 years ago
cskh	359a908bea	integ-test: remove unnecessary step since connection is already via mgw (#15381)	2 years ago
Derek Menteer	dc27e35f82	Consul 1.14 post-release updates (#15382) ... * Update changelog with 1.14 notes. * gomod version bumps for 1.14 release.	2 years ago
Derek Menteer	ef61bdf3c2	Remove unnecessary default test config. (#15361)	2 years ago
Kyle Schochenmaier	bf0f61a878	removes ioutil usage everywhere which was deprecated in go1.16 (#15297) ... * update go version to 1.18 for api and sdk, go mod tidy * removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward. Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	2 years ago
Freddy	7f5f7e9cf9	Avoid blocking child type updates on parent ack (#15083)	2 years ago
Chris S. Kim	985a4ee1b1	Update hcp-scada-provider to fix diamond dependency problem with go-msgpack (#15185)	2 years ago
R.B. Boyer	ed294fc49c	test: fix envoy integration tests to explicitly create config entries (#15269) ... This is instead of the current behavior where we feed the config entries in using the config_entries.bootstrap configuration which oddly races against other setup code in some circumstances. I converted ALL tests to explicitly create config entries.	2 years ago
Dan Stough	553312ef61	fix: persist peering CA updates to dialing clusters (#15243) ... fix: persist peering CA updates to dialing clusters	2 years ago
cskh	94d232ab1e	integ test: reduce flakiness due to compound output from retry (#15233) ... * integ test: avoid flakiness due to compound output from retry * changelog	2 years ago
Dan Stough	05e93f7569	test: refactor testcontainers and add peering integ tests (#15084)	2 years ago
Derek Menteer	2d4b62be3c	Add tests.	2 years ago
Chris S. Kim	41019adf56	Update go version to 1.19	2 years ago
cskh	fa970a85ad	integration test: specify image name for testing dev container (#15069) ... * integration test: specify image name for testing dev container * Add todo	2 years ago
cskh	6a9739ad7e	chore: fix the module path to align with the code structure (#15053)	2 years ago
R.B. Boyer	c3fa46c3f6	test: possibly fix flaky TestEnvoy/case-ingress-gateway-multiple-services test (#15034) ... The integration test TestEnvoy/case-ingress-gateway-multiple-services is flaky and this possibly reduces the flakiness by explicitly waiting for services to show up in the catalog as healthy before waiting for them to show up in envoy as healthy which gives it just a bit more time to sync.	2 years ago
freddygv	472a8e82dc	Add integ test for peering through gateways	2 years ago
freddygv	3034df6a5c	Require Connect and TLS to generate peering tokens ... By requiring Connect and a gRPC TLS listener we can automatically configure TLS for all peering control-plane traffic.	2 years ago
Eric Haberkorn	1633cf20ea	Make the mesh gateway changes to allow `local` mode for cluster peering data plane traffic (#14817) ... Make the mesh gateway changes to allow `local` mode for cluster peering data plane traffic	2 years ago
Alex Oskotsky	13da2c5fad	Add the ability to retry on reset connection to service-routers (#12890)	2 years ago
John Murret	79a541fd7d	Upgrade serf to v0.10.1 and memberlist to v0.5.0 to get memberlist size metrics and broadcast queue depth metric (#14873) ... * updating to serf v0.10.1 and memberlist v0.5.0 to get memberlist size metrics and memberlist broadcast queue depth metric * update changelog * update changelog * correcting changelog * adding "QueueCheckInterval" for memberlist to test * updating integration test containers to grab latest api	2 years ago
Derek Menteer	678adb3154	Add peering integration tests (#14836) ... Add peering integration tests.	2 years ago
Eric Haberkorn	1b565444be	Rename `PeerName` to `Peer` on prepared queries and exported services (#14854)	2 years ago
Luke Kysow	960c42854b	Remove terminal colouring from test output so it is (#14810) ... more readable in CI. ``` Running primary verification step for case-ingress-gateway-multiple-services... �[34;1mverify.bats �[0m�[1G ingress proxy admin is up on :20000�[K�[75G 1/12�[2G�[1G ✓ ingress proxy admin is up on :20000�[K �[0m�[1G s1 proxy admin is up on :19000�[K�[75G 2/12�[2G�[1G ✓ s1 proxy admin is up on :19000�[K �[0m�[1G s2 proxy admin is up on :19001�[K�[75G 3/12�[2G�[1G ✓ s2 proxy admin is up on :19001�[K �[0m�[1G s1 proxy listener should be up and have right cert�[K�[75G 4/12�[2G�[1G ✓ s1 proxy listener should be up and have right cert�[K �[0m�[1G s2 proxy listener should be up and have right cert�[K�[75G 5/12�[2G�[1G ✓ s2 proxy listener should be up and have right cert�[K �[0m�[1G ingress-gateway should have healthy endpoints for s1�[K�[75G 6/12�[2G�[31;1m�[1G ✗ ingress-gateway should have healthy endpoints for s1�[K �[0m�[31;22m (from function `assert_upstream_has_endpoints_in_status' in file /workdir/primary/bats/helpers.bash, line 385, ``` versus ``` Running primary verification step for case-ingress-gateway-multiple-services... 1..12 ok 1 ingress proxy admin is up on :20000 ok 2 s1 proxy admin is up on :19000 ok 3 s2 proxy admin is up on :19001 ok 4 s1 proxy listener should be up and have right cert ok 5 s2 proxy listener should be up and have right cert not ok 6 ingress-gateway should have healthy endpoints for s1 not ok 7 s1 proxy should have been configured with max_connections in services ok 8 ingress-gateway should have healthy endpoints for s2 ```	2 years ago
cskh	622804ad5b	fix flaky integration test (#14843)	2 years ago
cskh	69f40df548	feat(ingress gateway: support configuring limits in ingress-gateway c… (#14749) ... * feat(ingress gateway: support configuring limits in ingress-gateway config entry - a new Defaults field with max_connections, max_pending_connections, max_requests is added to ingress gateway config entry - new field max_connections, max_pending_connections, max_requests in individual services to overwrite the value in Default - added unit test and integration test - updated doc Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	2 years ago
DanStough	7704daaad5	release updates for 1.13.2, 1.12.5, and 1.11.9	2 years ago
Evan Culver	31ef39112a	Add more content to integration test docs (#14613)	2 years ago
Evan Culver	d0416f593c	connect: Bump latest Envoy to 1.23.1 in test matrix (#14573)	2 years ago
Luke Kysow	15043de647	Document integration tests (#14391)	2 years ago
Eric Haberkorn	aa8268e50c	Implement Cluster Peering Redirects (#14445) ... implement cluster peering redirects	2 years ago
Luke Kysow	095764a482	Suppress "unbound variable" error. (#14424) ... Without this change, you'd see this error: ``` ./run-tests.sh: line 49: LAMBDA_TESTS_ENABLED: unbound variable ./run-tests.sh: line 49: LAMBDA_TESTS_ENABLED: unbound variable ```	2 years ago
Eric Haberkorn	3726a0ab7a	Finish up cluster peering failover (#14396)	2 years ago
Luke Kysow	70bb6a2abd	Run integration tests locally using amd64 (#14365) ... Locally, always run integration tests using amd64, even if running on an arm mac. This ensures the architecture locally always matches the CI/CD environment. In addition: * Use consul:local for envoy integration and upgrade tests. Previously, consul:local was used for upgrade tests and consul-dev for integration tests. I didn't see a reason to use separate images as it's more confusing. * By default, disable the requirement that aws credentials are set. These are only needed for the lambda tests and make it so you can't run any tests locally, even if you're not running the lambda tests. Now they'll only run if the LAMBDA_TESTS_ENABLED env var is set. * Split out the building of the Docker image for integration tests into its own target from `dev-docker`. This allows us to always use an amd64 image without messing up the `dev-docker` target. * Add support for passing GO_TEST_FLAGs to `test-envoy-integ` target. * Add a wait_for_leader function because tests were failing locally without it.	2 years ago
Eric Haberkorn	ebd5513d4b	Refactor failover code to use Envoy's aggregate clusters (#14178)	2 years ago
Chris S. Kim	670531f828	Retry docker build steps	2 years ago
Luke Kysow	988e1fd35d	peering: default to false (#13963) ... * defaulting to false because peering will be released as beta * Ignore peering disabled error in bundles cachetype Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: freddygv <freddy@hashicorp.com> Co-authored-by: Matt Keeler <mjkeeler7@gmail.com>	2 years ago
Chris S. Kim	8ed49ea4d0	Update envoy metrics label extraction for peered clusters and listeners (#13818) ... Now that peered upstreams can generate envoy resources (#13758), we need a way to disambiguate local from peered resources in our metrics. The key difference is that datacenter and partition will be replaced with peer, since in the context of peered resources partition is ambiguous (could refer to the partition in a remote cluster or one that exists locally). The partition and datacenter of the proxy will always be that of the source service. Regexes were updated to make emitting datacenter and partition labels mutually exclusive with peer labels. Listener filter names were updated to better match the existing regex. Cluster names assigned to peered upstreams were updated to be synthesized from local peer name (it previously used the externally provided primary SNI, which contained the peer name from the other side of the peering). Integration tests were updated to assert for the new peer labels.	2 years ago
Chris Thain	af40b9b144	Add Consul Lambda integration tests (#13770)	2 years ago
Evan Culver	4116537b83	connect: Add support for Envoy 1.23, remove 1.19 (#13807)	2 years ago
R.B. Boyer	bb4d4040fb	server: ensure peer replication can successfully use TLS over external gRPC (#13733) ... Ensure that the peer stream replication rpc can successfully be used with TLS activated. Also: - If key material is configured for the gRPC port but HTTPS is not enabled now TLS will still be activated for the gRPC port. - peerstream replication stream opened by the establishing-side will now ignore grpc.WithBlock so that TLS errors will bubble up instead of being awkwardly delayed or suppressed	2 years ago
Evan Culver	d523d005d9	Latest submodule versions (#13750)	2 years ago
R.B. Boyer	af04851637	peering: move peer replication to the external gRPC port (#13698) ... Peer replication is intended to be between separate Consul installs and effectively should be considered "external". This PR moves the peer stream replication bidirectional RPC endpoint to the external gRPC server and ensures that things continue to function.	2 years ago
R.B. Boyer	1a9c86ea8f	xds: mesh gateways now correctly load up peer-exported discovery chains using L7 protocols (#13624) ... A mesh gateway will now configure the filter chains for L7 exported services using the correct discovery chain information.	2 years ago
R.B. Boyer	7133ee38f6	test: for upgrade compatibility tests retain assigned container ip addresses on upgrade (#13615) ... Use a synthetic pod construct to hold onto the IP address in the interim.	2 years ago
Dan Upton	ebf74d08fd	test: run Envoy integration tests against both servers and clients (#13610)	2 years ago
R.B. Boyer	a1e911a70c	tests: ensure integration tests show logs from the containers to help debugging (#13593)	2 years ago
Dhia Ayachi	355cbfa766	update github.com/containerd/containerd to 1.5.13 (#13520)	2 years ago
Evan Culver	7f8c650d61	connect: Use Envoy 1.22.2 instead of 1.22.1 (#13444)	2 years ago
Evan Culver	ba6136eb42	connect: Update Envoy support matrix to latest patch releases (#13431)	2 years ago
R.B. Boyer	7001e1151c	peering: rename initiate to establish in the context of the APIs (#13419)	2 years ago
R.B. Boyer	bba3eb8cdd	peering: mesh gateways are required for cross-peer service mesh communication (#13410) ... Require use of mesh gateways in order for service mesh data plane traffic to flow between peers. This also adds plumbing for envoy integration tests involving peers, and one starter peering test.	3 years ago
R.B. Boyer	fbee9eda08	test: break dep on main consul module (#13373) ... The main consul module is not a great library and complicates some oss/ent module issues. This undoes #13371	3 years ago