Commit Graph

324 Commits (51d8eb8e07f3ecf251380c90079ad7f984c734e3)

Author SHA1 Message Date
John Maguire fa93a0d4f7
Add namespace file with build tag for OSS gateway tests (#16590)
* Add namespace file with build tag for OSS tests

* Remove TODO comment
2023-03-09 20:46:02 +00:00
Semir Patel 9a5cb20570
Bump submodules from latest 1.15.1 patch release (#16578)
* Update changelog with Consul patch releases 1.13.7, 1.14.5, 1.15.1

* Bump submodules from latest patch release

* Forgot one
2023-03-08 14:37:50 -06:00
Anita Akaeze 280bdd3ea0
NET-2954: Improve integration tests CI execution time (#16565)
* NET-2954: Improve integration tests CI execution time

* fix ci

* remove comments and modify config file
2023-03-08 11:00:23 -05:00
cskh b649a5e8e4
upgrade test: use retry with ModifyIndex and remove ent test file (#16553) 2023-03-07 13:27:47 -05:00
cskh 94ecb9c5d5
upgrade test: discovery chain across partition (#16543) 2023-03-06 13:28:02 -05:00
Anita Akaeze 9d8e00db24
Merge pull request #4573 from hashicorp/NET-2841 (#16544)
* Merge pull request #4573 from hashicorp/NET-2841

NET-2841: PART 2 refactor upgrade tests to include version 1.15

* update upgrade versions
2023-03-06 11:40:33 -05:00
Melisa Griffin fc232326a0
NET-2904 Fixes API Gateway Route Service Weight Division Error 2023-03-06 08:41:57 -05:00
Andrew Stucki cc0765b87d
Fix resolution of service resolvers with subsets for external upstreams (#16499)
* Fix resolution of service resolvers with subsets for external upstreams

* Add tests

* Add changelog entry

* Update view filter logic
2023-03-03 14:17:11 -05:00
Andrew Stucki 5deffbd95b
Fix issue where terminating gateway service resolvers weren't properly cleaned up (#16498)
* Fix issue where terminating gateway service resolvers weren't properly cleaned up

* Add integration test for cleaning up resolvers

* Add changelog entry

* Use state test and drop integration test
2023-03-03 09:56:57 -05:00
Anita Akaeze 358c35ef70
Merge pull request #4584 from hashicorp/refactor_cluster_config (#16517)
NET-2841: PART 1 - refactor NewPeeringCluster to support custom config
2023-03-02 18:21:25 -05:00
Anita Akaeze 2b6d35fa8f
Merge pull request #4538 from hashicorp/NET-2396 (#16516)
NET-2396: refactor test to reduce duplication
2023-03-02 17:40:07 -05:00
Nick Irvine 367a64f059
NET-2292: port ingress-gateway test case "http" from BATS addendum (#16490) 2023-03-01 12:45:27 -08:00
sarahalsmiller 6db445ba29
Gateway Test HTTPPathRewrite (#16418)
* add http url path rewrite

* add Mike's test back in

* update kind to use api.APIGateway
2023-02-28 20:15:40 +00:00
Mike Morris 29db217a0e
gateways: add e2e test for API Gateway HTTPRoute ParentRef change (#16408)
* test(gateways): add API Gateway HTTPRoute ParentRef change test

* test(gateways): add checkRouteError helper

* test(gateways): remove EOF check

in CI this seems to sometimes be 'connection reset by peer' instead

* Update test/integration/consul-container/test/gateways/http_route_test.go
2023-02-28 13:57:29 -05:00
cskh c7713462ca
upgrade test: consolidate resolver test cases (#16443) 2023-02-27 20:38:31 +00:00
sarahalsmiller d99dcd48c2
Basic gobased API gateway spinup test (#16278)
* wip, proof of concept, gateway service being registered, don't know how to hit it

* checkpoint

* Fix up API Gateway go tests (#16297)

* checkpoint, getting InvalidDiscoveryChain route protocol does not match targeted service protocol

* checkpoint

* httproute hittable

* tests working, one header test failing

* differentiate services by status code, minor cleanup

* working tests

* updated GetPort interface

* fix getport

---------

Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-02-24 15:57:44 -05:00
Anita Akaeze 94b378998f
NO_JIRA: refactor validate function in traffic mgt tests (#16422) 2023-02-24 14:34:14 -05:00
Semir Patel d4dee31503
Try DRYing up createCluster in integration tests (#16199) 2023-02-23 16:51:20 -06:00
Anita Akaeze 1ed70cf9d0
NET-2286: Add tests to verify traffic redirects between services (#16390) 2023-02-23 17:28:42 -05:00
cskh 3de9f7fc17
upgrade test: peering with resolver and failover (#16391) 2023-02-23 12:53:56 -05:00
Eric Haberkorn 595131fca9
Refactor the disco chain -> xds logic (#16392) 2023-02-23 11:32:32 -05:00
Anita Akaeze 98a771d1e4
NET-2285: Assert total number of expected instances by Consul (#16371) 2023-02-22 15:43:20 -05:00
Anita Akaeze 84c7b0066c
initial code (#16296) 2023-02-22 12:52:14 -05:00
Derek Menteer 5309f68bc0
Upgrade Alpine image to 3.17 (#16358) 2023-02-22 10:09:41 -06:00
cskh de17c7c26f
upgrade test: splitter and resolver config entry in peered cluster (#16356) 2023-02-22 10:22:25 -05:00
Andrew Stucki 18e2ee77ca
[API Gateway] Fix targeting service splitters in HTTPRoutes (#16350)
* [API Gateway] Fix targeting service splitters in HTTPRoutes

* Fix test description
2023-02-22 03:48:26 +00:00
Andrew Stucki 7f9ec78932
[API Gateway] Validate listener name is not empty (#16340)
* [API Gateway] Validate listener name is not empty

* Update docstrings and test
2023-02-21 14:12:19 -05:00
wangxinyi7 9d55cd1f18
fix flakieness (#16338) 2023-02-21 08:47:11 -08:00
Dan Stough f1436109ea
[OSS] security: update go to 1.20.1 (#16263)
* security: update go to 1.20.1
2023-02-17 15:04:12 -05:00
Andrew Stucki e4a992c581
Fix hostname alignment checks for HTTPRoutes (#16300)
* Fix hostname alignment checks for HTTPRoutes
2023-02-17 18:18:11 +00:00
Andrew Stucki b3ddd4d24e
Inline API Gateway TLS cert code (#16295)
* Include secret type when building resources from config snapshot

* First pass at generating envoy secrets from api-gateway snapshot

* Update comments for xDS update order

* Add secret type + corresponding golden files to existing tests

* Initialize test helpers for testing api-gateway resource generation

* Generate golden files for new api-gateway xDS resource test

* Support ADS for TLS certificates on api-gateway

* Configure TLS on api-gateway listeners

* Inline TLS cert code

* update tests

* Add SNI support so we can have multiple certificates

* Remove commented out section from helper

* regen deep-copy

* Add tcp tls test

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-02-17 12:46:03 -05:00
Nitya Dhanushkodi 8dab825c36
troubleshoot: fixes and updated messages (#16294) 2023-02-17 07:43:05 -08:00
Thomas Eckert 2460ac99c9
API Gateway Envoy Golden Listener Tests (#16221)
* Simple API Gateway e2e test for tcp routes

* Drop DNSSans since we don't front the Gateway with a leaf cert

* WIP listener tests for api-gateway

* Return early if no routes

* Add back in leaf cert to testing

* Fix merge conflicts

* Re-add kind to setup

* Fix iteration over listener upstreams

* New tcp listener test

* Add tests for API Gateway with TCP and HTTP routes

* Move zero-route check back

* Drop generateIngressDNSSANs

* Check for chains not routes

---------

Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-02-16 14:42:36 -05:00
cskh fd61605ffb
upgrade test: fix flaky peering through mesh gateway (#16271) 2023-02-15 10:26:43 -05:00
malizz 247211de6a
add integration tests for troubleshoot (#16223)
* draft

* expose internal admin port and add proxy test

* update tests

* move comment

* add failure case, fix lint issues

* cleanup

* handle error

* revert changes to service interface

* address review comments

* fix merge conflict

* merge the tests so cluster is created once

* fix other test
2023-02-14 14:22:09 -08:00
cskh 5b7f36c2ce
integ test: fix retry upstream test (#16246) 2023-02-13 15:16:56 -05:00
Andrew Stucki 9bb0ecfc18
[API Gateway] Add integration test for HTTP routes (#16236)
* [API Gateway] Add integration test for conflicted TCP listeners

* [API Gateway] Update simple test to leverage intentions and multiple listeners

* Fix broken unit test

* [API Gateway] Add integration test for HTTP routes
2023-02-13 14:18:05 -05:00
cskh ab5dac3414
upgrade test: peering with http router config entry (#16231)
* upgrade test: peering with http router config entry
2023-02-13 14:09:12 -05:00
Andrew Stucki 8ff2974dbe
[API Gateway] Update simple test to leverage intentions and multiple listeners (#16228)
* [API Gateway] Add integration test for conflicted TCP listeners

* [API Gateway] Update simple test to leverage intentions and multiple listeners

* Fix broken unit test

* PR suggestions
2023-02-10 21:13:44 +00:00
Andrew Stucki 318ba215ab
[API Gateway] Add integration test for conflicted TCP listeners (#16225) 2023-02-10 11:34:01 -06:00
Derek Menteer 4f2ce60654
Fix peering acceptors in secondary datacenters. (#16230)
Prior to this commit, secondary datacenters could not be initialized
as peering acceptors if ACLs were enabled. This is due to the fact that
internal server-to-server API calls would fail because the management
token was not generated. This PR makes it so that both primary and
secondary datacenters generate their own management token whenever
a leader is elected in their respective clusters.
2023-02-10 09:47:17 -06:00
Andrew Stucki 3b9c569561
Simple API Gateway e2e test for tcp routes (#16222)
* Simple API Gateway e2e test for tcp routes

* Drop DNSSans since we don't front the Gateway with a leaf cert
2023-02-09 16:20:12 -05:00
Andrew Stucki f4210d47dd
Add basic smoke test to make sure an APIGateway runs (#16217) 2023-02-09 11:32:10 -05:00
Anita Akaeze d72ad5fb95
Merge pull request #4216 from hashicorp/NET-2252-add-assert-fortioname (#16212)
NET-2252: integration tests: add assert.FortioName
2023-02-09 09:45:31 -05:00
cskh e91bc9c058
feat: envoy extension - http local rate limit (#16196)
- http local rate limit
- Apply rate limit only to local_app
- unit test and integ test
2023-02-07 21:56:15 -05:00
cskh b4151780d6
Upgrade test: verify the agent token is working after upgrade (#16164)
1. Upgraded agent can inherit the persisted token and join the cluster
2. Agent token prior to upgrade is still valid after upgrade
3. Enable ACL in the agent configuration
2023-02-07 14:13:19 -05:00
wangxinyi7 906ebb97f6
change log level (#16128) 2023-02-06 12:58:13 -08:00
Anita Akaeze f58d5bb221
NET-2087: Restart proxy sidecar during cluster upgrade (#16140) 2023-02-06 13:09:44 -05:00
Anita Akaeze fcf18222c5
add assertions (#16087) 2023-02-03 10:20:22 -05:00
Dan Upton e40b731a52
rate: add prometheus definitions, docs, and clearer names (#15945) 2023-02-03 12:01:57 +00:00
Anita Akaeze 08a19e532d
NO_JIRA: Add function to get container status before making api call (#16116) 2023-02-01 10:48:54 -05:00
cskh f6da81c9d0
improvement: prevent filter being added twice from any enovy extension (#16112)
* improvement: prevent filter being added twice from any enovy extension

* break if error != nil

* update test
2023-01-31 16:49:45 +00:00
cskh 9da61c1cec
Upgrade test: retain sidecar containers during upgrade. (#16100) 2023-01-30 09:49:52 -05:00
cskh ffb81782de
Upgrade test: peering control plane traffic through mesh gateway (#16091) 2023-01-27 11:25:48 -05:00
cskh 5fa9ab28dc
integ test: remove hardcoded upstream local bind port and max number of envoy sidecar (#16092) 2023-01-27 15:19:10 +00:00
cskh ebdb8e5fb2
flaky test: use retry long to wait for config entry upgrade (#16068)
* flaky test: use retry long to wait for config entry upgrade

* increase wait for rbac policy
2023-01-26 11:01:17 -05:00
cskh dbaab52786
Post upgrade test validation: envoy endpoint and register service (#16067) 2023-01-25 12:27:36 -05:00
Dan Stough bb71d045e1
test: run integration tests in parallel (#16035) 2023-01-24 14:51:50 -05:00
R.B. Boyer 96389eb266
test: container tests wait for available networks (#16045) 2023-01-23 14:14:24 -06:00
Dan Stough 91d6a81c14
test(integration): add access logging test (#16008) 2023-01-20 17:02:44 -05:00
John Murret 794277371f
Integration test for server rate limiting (#15960)
* rate limit test

* Have tests for the 3 modes

* added assertions for logs and metrics

* add comments to test sections

* add check for rate limit exceeded text in log assertion section.

* fix linting error

* updating test to use KV get and put.  move log assertion tolast.

* Adding logging for blocking messages in enforcing mode.  refactoring tests.

* modified test description

* formatting

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* Update test/integration/consul-container/test/ratelimit/ratelimit_test.go

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

* expand log checking so that it ensures both logs are they when they are supposed to be and not there when they are not expected to be.

* add retry on test

* Warn once when rate limit exceed regardless of enforcing vs permissive.

* Update test/integration/consul-container/test/ratelimit/ratelimit_test.go

Co-authored-by: Dan Upton <daniel@floppy.co>

Co-authored-by: Dan Upton <daniel@floppy.co>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2023-01-19 08:43:33 -07:00
Anita Akaeze 4e154144a6
NET-2038: Add envoy assertion function of listener verification (#15969) 2023-01-18 16:13:55 -05:00
Dan Stough e8dde59bd0
chore(ci): fix compat ent compat tests for sidecars and gateways (#15997) 2023-01-17 17:16:55 -05:00
R.B. Boyer d59efd390c
test: general cleanup and fixes for the container integration test suite (#15959)
- remove dep on consul main module
- use 'consul tls' subcommands instead of tlsutil
- use direct json config construction instead of agent/config structs
- merge libcluster and libagent packages together
- more widely use BuildContext
- get the OSS/ENT runner stuff working properly
- reduce some flakiness
- fix some correctness related to http/https API
2023-01-11 15:34:27 -06:00
Dan Stough 6d2880e894
feat: add access logs to dataplane bootstrap rpc (#15951) 2023-01-11 13:40:09 -05:00
Matt Keeler 5afd4657ec
Protobuf Modernization (#15949)
* Protobuf Modernization

Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf

Marshallers (protobuf and json) needed some changes to account for different APIs.

Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package.

This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces.

* Fix go-mod-tidy make target to work on all modules
2023-01-11 09:39:10 -05:00
Eric Haberkorn 8d923c1789
Add the Lua Envoy extension (#15906) 2023-01-06 12:13:40 -05:00
cskh 1113314641
Refactoring the peering integ test to accommodate coming changes of o… (#15885)
* Refactoring the peering integ test to accommodate coming changes of other upgrade scenarios.

- Add a utils package under test that contains methods to set up various test scenarios.
- Deduplication: have a single CreatingPeeringClusterAndSetup replace
  CreatingAcceptingClusterAndSetup and CreateDialingClusterAndSetup.
- Separate peering cluster creation and server registration.

* Apply suggestions from code review

Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
2023-01-04 15:28:15 -05:00
Dan Upton d4c435856b
grpc: `protoc` plugin for generating gRPC rate limit specifications (#15564)
Adds automation for generating the map of `gRPC Method Name → Rate Limit Type`
used by the middleware introduced in #15550, and will ensure we don't forget
to add new endpoints.

Engineers must annotate their RPCs in the proto file like so:

```
rpc Foo(FooRequest) returns (FooResponse) {
  option (consul.internal.ratelimit.spec) = {
    operation_type: READ,
  };
}
```

When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will
be installed that writes rate-limit specs as a JSON array to a file called
`.ratelimit.tmp` (one per protobuf package/directory).

After running Buf, `make proto` will execute a post-process script that will
ingest all of the `.ratelimit.tmp` files and generate a Go file containing the
mappings in the `agent/grpc-middleware` package. In the enterprise repository,
it will write an additional file with the enterprise-only endpoints.

If an engineer forgets to add the annotation to a new RPC, the plugin will
return an error like so:

```
RPC Foo is missing rate-limit specification, fix it with:

	import "proto-public/annotations/ratelimit/ratelimit.proto";

	service Bar {
	  rpc Foo(...) returns (...) {
	    option (hashicorp.consul.internal.ratelimit.spec) = {
	      operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT,
	    };
	  }
	}
```

In the future, this annotation can be extended to support rate-limit
category (e.g. KV vs Catalog) and to determine the retry policy.
2023-01-04 16:07:02 +00:00
Nitya Dhanushkodi d382ca0aec
extensions: refactor serverless plugin to use extensions from config entry fields (#15817)
docs: update config entry docs and the Lambda manual registration docs

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
Co-authored-by: Eric <eric@haberkorn.co>
2022-12-19 12:19:37 -08:00
cskh 692a6edd7d
Upgrade test: test peering upgrade from an old version of consul (#15768)
* upgrade test: test peering upgrade from an old version of consul

NET-1809
2022-12-15 16:31:12 -05:00
Semir Patel bafa5c7156
Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
cskh 04bf24c8c1
feat(ingress-gateway): support outlier detection of upstream service for ingress gateway (#15614)
* feat(ingress-gateway): support outlier detection of upstream service for ingress gateway

* changelog

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
2022-12-13 11:51:37 -05:00
Dan Stough 98ef5f28dd
[OSS] security: update x/net module (#15737)
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-12-08 16:45:44 -05:00
Eric Haberkorn 4268c1c25c
Remove the `connect.enable_serverless_plugin` agent configuration option (#15710) 2022-12-08 14:46:42 -05:00
cskh 3df68751f5
Flakiness test: case-cfg-splitter-peering-ingress-gateways (#15707)
* integ-test: fix flaky test - case-cfg-splitter-peering-ingress-gateways

* add retry peering to all peering cases

Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
2022-12-07 20:19:34 -05:00
cskh 36f05bc8fb
integ-test: test consul upgrade from the snapshot of a running cluster (#15595)
* integ-test: test consul upgrade from the snapshot of a running cluster

* use Target version as default


Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
2022-12-01 10:39:09 -05:00
Dan Stough 227fd14287
chore: updates from 1.14.2 release (#15633)
* chore: updates from 1.14.2 release
2022-11-30 22:15:58 -05:00
Derek Menteer 8079686bf0
Add 1.14.1 release updates. (#15514)
Add post-release changes for 1.14.1 updates.
2022-11-21 13:35:30 -06:00
cskh 359a908bea
integ-test: remove unnecessary step since connection is already via mgw (#15381) 2022-11-15 15:26:40 -05:00
Derek Menteer dc27e35f82
Consul 1.14 post-release updates (#15382)
* Update changelog with 1.14 notes.

* gomod version bumps for 1.14 release.
2022-11-15 14:22:43 -06:00
Derek Menteer ef61bdf3c2
Remove unnecessary default test config. (#15361) 2022-11-14 14:07:42 -06:00
Kyle Schochenmaier bf0f61a878
removes ioutil usage everywhere which was deprecated in go1.16 (#15297)
* update go version to 1.18 for api and sdk, go mod tidy
* removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward.
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2022-11-10 10:26:01 -06:00
Freddy 7f5f7e9cf9
Avoid blocking child type updates on parent ack (#15083) 2022-11-07 18:10:42 -07:00
Chris S. Kim 985a4ee1b1
Update hcp-scada-provider to fix diamond dependency problem with go-msgpack (#15185) 2022-11-07 11:34:30 -05:00
R.B. Boyer ed294fc49c
test: fix envoy integration tests to explicitly create config entries (#15269)
This is instead of the current behavior where we feed the config entries in using the config_entries.bootstrap configuration which oddly races against other setup code in some circumstances.

I converted ALL tests to explicitly create config entries.
2022-11-07 10:02:04 -06:00
Dan Stough 553312ef61
fix: persist peering CA updates to dialing clusters (#15243)
fix: persist peering CA updates to dialing clusters
2022-11-04 12:53:20 -04:00
cskh 94d232ab1e
integ test: reduce flakiness due to compound output from retry (#15233)
* integ test: avoid flakiness due to compound output from retry

* changelog
2022-11-02 14:08:17 -04:00
Dan Stough 05e93f7569
test: refactor testcontainers and add peering integ tests (#15084) 2022-11-01 15:03:23 -04:00
Derek Menteer 2d4b62be3c Add tests. 2022-10-31 08:45:00 -05:00
Chris S. Kim 41019adf56 Update go version to 1.19 2022-10-24 16:12:08 -04:00
cskh fa970a85ad
integration test: specify image name for testing dev container (#15069)
* integration test: specify image name for testing dev container

* Add todo
2022-10-20 17:12:36 -04:00
cskh 6a9739ad7e
chore: fix the module path to align with the code structure (#15053) 2022-10-19 10:36:35 -04:00
R.B. Boyer c3fa46c3f6
test: possibly fix flaky TestEnvoy/case-ingress-gateway-multiple-services test (#15034)
The integration test TestEnvoy/case-ingress-gateway-multiple-services is flaky
and this possibly reduces the flakiness by explicitly waiting for services to show
up in the catalog as healthy before waiting for them to show up in envoy as
healthy which gives it just a bit more time to sync.
2022-10-18 14:23:52 -05:00
freddygv 472a8e82dc Add integ test for peering through gateways 2022-10-13 14:58:05 -06:00
freddygv 3034df6a5c Require Connect and TLS to generate peering tokens
By requiring Connect and a gRPC TLS listener we can automatically
configure TLS for all peering control-plane traffic.
2022-10-07 09:06:29 -06:00
Eric Haberkorn 1633cf20ea
Make the mesh gateway changes to allow `local` mode for cluster peering data plane traffic (#14817)
Make the mesh gateway changes to allow `local` mode for cluster peering data plane traffic
2022-10-06 09:54:14 -04:00
Alex Oskotsky 13da2c5fad
Add the ability to retry on reset connection to service-routers (#12890) 2022-10-05 13:06:44 -04:00
John Murret 79a541fd7d
Upgrade serf to v0.10.1 and memberlist to v0.5.0 to get memberlist size metrics and broadcast queue depth metric (#14873)
* updating to serf v0.10.1 and memberlist v0.5.0 to get memberlist size metrics and memberlist broadcast queue depth metric

* update changelog

* update changelog

* correcting changelog

* adding "QueueCheckInterval" for memberlist to test

* updating integration test containers to grab latest api
2022-10-04 17:51:37 -06:00
Derek Menteer 678adb3154
Add peering integration tests (#14836)
Add peering integration tests.
2022-10-04 13:51:04 -05:00