244 Commits (d574473fd104bd14335c716e179464433e637352)

Author	SHA1	Message	Date
Blake Covarrubias	019c62e1ba	xds: Use downstream protocol when connecting to local app (#18573) ... Configure Envoy to use the same HTTP protocol version used by the downstream caller when forwarding requests to a local application that is configured with the protocol set to either `http2` or `grpc`. This allows upstream applications that support both HTTP/1.1 and HTTP/2 on a single port to receive requests using either protocol. This is beneficial when the application primarily communicates using HTTP/2, but also needs to support HTTP/1.1, such as to respond to Kubernetes HTTP readiness/liveness probes. Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>	1 year ago
Eric Haberkorn	12be06f8e5	Add V2 TCP traffic permissions (#18771) ... Add support for TCP traffic permissions	1 year ago
Chris Thain	4724a4e169	Add Envoy golden test for OTEL access logging extension (#18760)	1 year ago
R.B. Boyer	a69e901660	xds: update golden tests to be deterministic (#18707)	1 year ago
John Maguire	2c244b6f42	[APIGW] NET-5017 JWT Cleanup/Status Conditions (#18700) ... * Fixes issues in setting status * Update golden files for changes to xds generation to not use deprecated methods * Fixed default for validation of JWT for route	1 year ago
sarahalsmiller	e235c8be3c	NET-5115 Add retry + timeout filters for api-gateway (#18324) ... * squash, implement retry/timeout in consul core * update tests	1 year ago
Dan Stough	284e3bdb54	[OSS] test: xds coverage for routes (#18369) ... test: xds coverage for routes	1 year ago
Dan Stough	8e3a1ddeb6	[OSS] Improve xDS Code Coverage - Endpoints and Misc (#18222) ... test: improve xDS endpoints code coverage	1 year ago
Dan Stough	2793761702	[OSS] Improve xDS Code Coverage - Clusters (#18165) ... test: improve xDS cluster code coverage	1 year ago
Blake Covarrubias	2c5a09bb0a	Explicitly enable WebSocket upgrades (#18150) ... This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283	1 year ago
Ronald	18bc04165c	Improve XDS test coverage: JWT auth edition (#18183) ... * Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>	1 year ago
Dan Stough	33d898b857	[OSS] test: improve xDS listener code coverage (#18138) ... test: improve xDS listener code coverage	1 year ago
Ronald	bcc6a9d752	Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062) ... ### Description <!-- Please describe why you're making this change, in plain English. --> - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added	1 year ago
Ronald	80394278b8	Expose JWKS cluster config through JWTProviderConfigEntry (#17978) ... * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA	1 year ago
Chris S. Kim	50a9d1b696	Remove POC code (#17974)	1 year ago
Ronald	1512ea307e	Dynamically create jwks clusters for jwt-providers (#17944)	1 year ago
Chris Thain	366bd6f89f	ext-authz Envoy extension: support `localhost` as a valid target URI. (#17821)	1 year ago
Paul Glass	d2363eb711	Test permissive mTLS filter chain not configured with tproxy disabled (#17747)	1 year ago
Chris Thain	a8f1350835	ENT merge of ext-authz extension updates (#17684)	1 year ago
Nathan Coleman	1074252361	api-gateway: stop adding all header filters to virtual host when generating xDS (#17644) ... * Add header filter to api-gateway xDS golden test * Stop adding all header filters to virtual host when generating xDS for api-gateway * Regenerate xDS golden file for api-gateway w/ header filter	1 year ago
Eric Haberkorn	779647b948	Add Envoy and Consul version constraints to Envoy extensions (#17612)	1 year ago
Michael Zalimeni	ad03a5d0f2	Avoid panic applying TProxy Envoy extensions (#17537) ... When UpstreamEnvoyExtender was introduced, some code was left duplicated between it and BasicEnvoyExtender. One path in that code panics when a TProxy listener patch is attempted due to no upstream data in RuntimeConfig matching the local service (which would only happen in rare cases). Instead, we can remove the special handling of upstream VIPs from BasicEnvoyExtender entirely, greatly simplifying the listener filter patch code and avoiding the panic. UpstreamEnvoyExtender, which needs this code to function, is modified to ensure a panic does not occur. This also fixes a second regression in which the Lua extension was not applied to TProxy outbound listeners.	2 years ago
Eric Haberkorn	d99312b86e	Add Upstream Service Targeting to Property Override Extension (#17517) ... * add upstream service targeting to property override extension * Also add baseline goldens for service specific property override extension. * Refactor the extension framework to put more logic into the templates. * fix up the golden tests	2 years ago
Ronald	55e283dda9	[NET-3092] JWT Verify claims handling (#17452) ... * [NET-3092] JWT Verify claims handling	2 years ago
Chris Thain	65b8ccdc1b	Enable Network filters for Wasm Envoy Extension (#17505)	2 years ago
Michael Zalimeni	e1df0f28bd	Support `Listener` and `ClusterLoadAssignment` in `property-override` (#17497) ... * Support Listener in Property Override Add support for patching `Listener` resources via the builtin `property-override` extension. Refactor existing listener patch code in `BasicEnvoyExtender` to simplify addition of resource support. * Support ClusterLoadAssignment in Property Override Add support for patching `ClusterLoadAssignment` resources via the builtin `property-override` extension.	2 years ago
Michael Zalimeni	5a46a8c604	Add `builtin/property-override` Envoy Extension (#17487) ... `property-override` is an extension that allows for arbitrarily patching Envoy resources based on resource matching filters. Patch operations resemble a subset of the JSON Patch spec with minor differences to facilitate patching pre-defined (protobuf) schemas. See Envoy Extension product documentation for more details. Co-authored-by: Eric Haberkorn <eric.haberkorn@hashicorp.com> Co-authored-by: Kyle Havlovitz <kyle@hashicorp.com>	2 years ago
Chris Thain	516eb4febc	Add `builtin/ext-authz` Envoy Extension (#17495)	2 years ago
Paul Glass	07ff9d3d64	Use original_dst filter instead of use_original_dst field (#17433)	2 years ago
Ronald	ddb25cec0e	[NET-3092] Improve jwt-provider tests (#17430) ... * [NET-3092] more tests, prior to verify claims work	2 years ago
Dan Stough	d935c7b466	[OSS] gRPC Blocking Queries (#17426) ... * feat: initial grpc blocking queries * changelog and docs update	2 years ago
Michael Zalimeni	b8d2640429	Disable remote proxy patching except AWS Lambda (#17415) ... To avoid unintended tampering with remote downstreams via service config, refactor BasicEnvoyExtender and RuntimeConfig to disallow typical Envoy extensions from being applied to non-local proxies. Continue to allow this behavior for AWS Lambda and the read-only Validate builtin extensions. Addresses CVE-2023-2816.	2 years ago
Ronald	113202d541	JWT Authentication with service intentions: xds package update (#17414) ... * JWT Authentication with service intentions: update xds package to translate config to envoy	2 years ago
Kyle Havlovitz	2904d0a431	Pull virtual IPs for filter chains from discovery chains (#17375)	2 years ago
Connor	0789661ce5	Rename hcp-metrics-collector to consul-telemetry-collector (#17327) ... * Rename hcp-metrics-collector to consul-telemetry-collector * Fix docs * Fix doc comment --------- Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>	2 years ago
Freddy	7c3e9cd862	Hash namespace+proxy ID when creating socket path (#17204) ... UNIX domain socket paths are limited to 104-108 characters, depending on the OS. This limit was quite easy to exceed when testing the feature on Kubernetes, due to how proxy IDs encode the Pod ID eg: metrics-collector-59467bcb9b-fkkzl-hcp-metrics-collector-sidecar-proxy To ensure we stay under that character limit this commit makes a couple changes: - Use a b64 encoded SHA1 hash of the namespace + proxy ID to create a short and deterministic socket file name. - Add validation to proxy registrations and proxy-defaults to enforce a limit on the socket directory length.	2 years ago
Semir Patel	5eaeb7b8e5	Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks (#15979) ... * Add MaxEjectionPercent to config entry * Add BaseEjectionTime to config entry * Add MaxEjectionPercent and BaseEjectionTime to protobufs * Add MaxEjectionPercent and BaseEjectionTime to api * Fix integration test breakage * Verify MaxEjectionPercent and BaseEjectionTime in integration test upstream confings * Website docs for MaxEjectionPercent and BaseEjection time * Add `make docs` to browse docs at http://localhost:3000 * Changelog entry * so that is the difference between consul-docker and dev-docker * blah * update proto funcs * update proto --------- Co-authored-by: Maliz <maliheh.monshizadeh@hashicorp.com>	2 years ago
Paul Glass	77ecff3209	Permissive mTLS (#17035) ... This implements permissive mTLS , which allows toggling services into "permissive" mTLS mode. Permissive mTLS mode allows incoming "non Consul-mTLS" traffic to be forward unmodified to the application. * Update service-defaults and proxy-defaults config entries with a MutualTLSMode field * Update the mesh config entry with an AllowEnablingPermissiveMutualTLS field and implement the necessary validation. AllowEnablingPermissiveMutualTLS must be true to allow changing to MutualTLSMode=permissive, but this does not require that all proxy-defaults and service-defaults are currently in strict mode. * Update xDS listener config to add a "permissive filter chain" when MutualTLSMode=permissive for a particular service. The permissive filter chain matches incoming traffic by the destination port. If the destination port matches the service port from the catalog, then no mTLS is required and the traffic sent is forwarded unmodified to the application.	2 years ago
Chris Thain	175bb1a303	Wasm Envoy HTTP extension (#16877)	2 years ago
Derek Menteer	2236975011	Change partition for peers in discovery chain targets (#16769) ... This commit swaps the partition field to the local partition for discovery chains targeting peers. Prior to this change, peer upstreams would always use a value of default regardless of which partition they exist in. This caused several issues in xds / proxycfg because of id mismatches. Some prior fixes were made to deal with one-off id mismatches that this PR also cleans up, since they are no longer needed.	2 years ago
Eric Haberkorn	495ad4c7ef	add enterprise xds tests (#16738)	2 years ago
Andrew Stucki	501b87fd31	[API Gateway] Fix invalid cluster causing gateway programming delay (#16661) ... * Add test for http routes * Add fix * Fix tests * Add changelog entry * Refactor and fix flaky tests	2 years ago
Eric Haberkorn	57e034b746	fix confusing spiffe ids in golden tests (#16643)	2 years ago
Ashvitha	f95ffe0355	Allow HCP metrics collection for Envoy proxies ... Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com> Add a new envoy flag: "envoy_hcp_metrics_bind_socket_dir", a directory where a unix socket will be created with the name `<namespace>_<proxy_id>.sock` to forward Envoy metrics. If set, this will configure: - In bootstrap configuration a local stats_sink and static cluster. These will forward metrics to a loopback listener sent over xDS. - A dynamic listener listening at the socket path that the previously defined static cluster is sending metrics to. - A dynamic cluster that will forward traffic received at this listener to the hcp-metrics-collector service. Reasons for having a static cluster pointing at a dynamic listener: - We want to secure the metrics stream using TLS, but the stats sink can only be defined in bootstrap config. With dynamic listeners/clusters we can use the proxy's leaf certificate issued by the Connect CA, which isn't available at bootstrap time. - We want to intelligently route to the HCP collector. Configuring its addreess at bootstrap time limits our flexibility routing-wise. More on this below. Reasons for defining the collector as an upstream in `proxycfg`: - The HCP collector will be deployed as a mesh service. - Certificate management is taken care of, as mentioned above. - Service discovery and routing logic is automatically taken care of, meaning that no code changes are required in the xds package. - Custom routing rules can be added for the collector using discovery chain config entries. Initially the collector is expected to be deployed to each admin partition, but in the future could be deployed centrally in the default partition. These config entries could even be managed by HCP itself.	2 years ago
Andrew Stucki	4b661d1e0c	Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable (#16495) ... * Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable * Regenerate golden files * Add RequestTimeout field * Add changelog entry	2 years ago
Eric Haberkorn	595131fca9	Refactor the disco chain -> xds logic (#16392)	2 years ago
Andrew Stucki	b3ddd4d24e	Inline API Gateway TLS cert code (#16295) ... * Include secret type when building resources from config snapshot * First pass at generating envoy secrets from api-gateway snapshot * Update comments for xDS update order * Add secret type + corresponding golden files to existing tests * Initialize test helpers for testing api-gateway resource generation * Generate golden files for new api-gateway xDS resource test * Support ADS for TLS certificates on api-gateway * Configure TLS on api-gateway listeners * Inline TLS cert code * update tests * Add SNI support so we can have multiple certificates * Remove commented out section from helper * regen deep-copy * Add tcp tls test --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>	2 years ago
Thomas Eckert	2460ac99c9	API Gateway Envoy Golden Listener Tests (#16221) ... * Simple API Gateway e2e test for tcp routes * Drop DNSSans since we don't front the Gateway with a leaf cert * WIP listener tests for api-gateway * Return early if no routes * Add back in leaf cert to testing * Fix merge conflicts * Re-add kind to setup * Fix iteration over listener upstreams * New tcp listener test * Add tests for API Gateway with TCP and HTTP routes * Move zero-route check back * Drop generateIngressDNSSANs * Check for chains not routes --------- Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
cskh	e91bc9c058	feat: envoy extension - http local rate limit (#16196) ... - http local rate limit - Apply rate limit only to local_app - unit test and integ test	2 years ago
Nitya Dhanushkodi	8d4c3aa42c	refactor: move service to service validation to troubleshoot package (#16132) ... This is to reduce the dependency on xds from within the troubleshoot package.	2 years ago