603 Commits (cc518b86ff1e7457f535161fe97779419e54de25)

Author	SHA1	Message	Date
Tom Davies	31aae80389	Allow multiple endpoints in Envoy clusters configured with hostnames (#21655) ... * xds: allow multiple endpoints for strict_dns * xds: fixes typo in multi hostname warning	4 weeks ago
Michael Zalimeni	d9206fc7e2	[NET-1151 NET-11228] security: Add request normalization and header match options to prevent L7 intentions bypass (#21816) ... mesh: add options for HTTP incoming request normalization Expose global mesh configuration to enforce inbound HTTP request normalization on mesh traffic via Envoy xDS config. mesh: enable inbound URL path normalization by default mesh: add support for L7 header match contains and ignore_case Enable partial string and case-insensitive matching in L7 intentions header match rules. ui: support L7 header match contains and ignore_case Co-authored-by: Phil Renaud <phil@riotindustries.com> test: add request normalization integration bats tests Add both "positive" and "negative" test suites, showing normalization in action as well as expected results when it is not enabled, for the same set of test cases. Also add some alternative service container test helpers for verifying raw HTTP request paths, which is difficult to do with Fortio. docs: update security and reference docs for L7 intentions bypass prevention - Update security docs with best practices for service intentions configuration - Update configuration entry references for mesh and intentions to reflect new values and add guidance on usage	1 month ago
John Maguire	a3ac555a5e	[NET-10952] fix cluster dns lookup family to gracefully handle ipv6 (#21703) ... * update jwks cluster creation to gracefully handle ipv6 * update unit tests for dns lookup family * Add changelog	2 months ago
R.B. Boyer	3e6f1c1fe1	remove v2 tenancy, catalog, and mesh (#21592) ... * remove v2 tenancy, catalog, and mesh - Inline the v2tenancy experiment to false - Inline the resource-apis experiment to false - Inline the hcp-v2-resource-apis experiment to false - Remove ACL policy templates and rule language changes related to workload identities (a v2-only concept) (e.g. identity and identity_prefix) - Update the gRPC endpoint used by consul-dataplane to no longer respond specially for v2 - Remove stray v2 references scattered throughout the DNS v1.5 newer implementation. * changelog * go mod tidy on consul containers * lint fixes from ENT --------- Co-authored-by: John Murret <john.murret@hashicorp.com>	3 months ago
Nitya Dhanushkodi	ed738a6f98	fix: use Envoy's default for validate_clusters to fix breaking routes when some backend clusters don't exist (#21587)	3 months ago
John Maguire	58fad92cd3	fix where jwt clusters are generated (#21606)	3 months ago
John Maguire	1fa428552b	[NET-10719] Fix cluster generation for jwt clusters for external jwt providers (#21604) ... * Fix cluster generation for jwt clusters for external jwt providers * add changelog	3 months ago
John Maguire	8555404662	[NET-10733] fix generation of xds resources (#21603) ... fix generation of xds resources	3 months ago
Deniz Onur Duzgun	68a7648d14	security: resolve incorrect type conversions (#21251) ... * security: resolve incorrect type conversions * add changelog * fix more incorrect type conversions	6 months ago
Nathan Coleman	b5b3a63183	[NET-9098] Narrow scope of peering config on terminating gw filter chain to TCP services (#21054)	7 months ago
sarahalsmiller	08761f16c8	Net 6820 customize mesh gateway limits (#20945) ... * add upstream limits to mesh gateway cluster generation * changelog * go mod tidy * readd changelog data * undo reversion from rebase * run codegen * Update .changelog/20945.txt Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * address notes * gofmt * clean up * gofmt * Update agent/proxycfg/mesh_gateway.go * gofmt * nil check --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>	7 months ago
Nathan Coleman	5e9f02d4be	[NET-8091] Add file-system-certificate config entry for API gateway (#20873) ... * Define file-system-certificate config entry * Collect file-system-certificate(s) referenced by api-gateway onto snapshot * Add file-system-certificate to config entry kind allow lists * Remove inapplicable validation This validation makes sense for inline certificates since Consul server is holding the certificate; however, for file system certificates, Consul server never actually sees the certificate. * Support file-system-certificate as source for listener TLS certificate * Add more required mappings for the new config entry type * Construct proper TLS context based on certificate kind * Add support or SDS in xdscommon * Remove unused param * Adds back verification of certs for inline-certificates * Undo tangential changes to TLS config consumption * Remove stray curly braces * Undo some more tangential changes * Improve function name for generating API gateway secrets * Add changelog entry * Update .changelog/20873.txt Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> * Add some nil-checking, remove outdated TODO * Update test assertions to include file-system-certificate * Add documentation for file-system-certificate config entry Add new doc to nav * Fix grammar mistake * Rename watchmaps, remove outdated TODO --------- Co-authored-by: Melisa Griffin <melisa.griffin@hashicorp.com> Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>	7 months ago
Nathan Coleman	9af713ff17	[NET-5772] Make tcp external service registered on terminating gw reachable from peered cluster (#19881) ... * Include SNI + root PEMs from peered cluster on terminating gw filter chain This allows an external service registered on a terminating gateway to be exported to and reachable from a peered cluster * Abstract existing logic into re-usable function * Regenerate golden files w/ new listener logic * Add changelog entry * Use peering bundles that are stable across test runs	8 months ago
Derek Menteer	0ac8ae6c3b	Fix xDS deadlock due to syncLoop termination. (#20867) ... * Fix xDS deadlock due to syncLoop termination. This fixes an issue where agentless xDS streams can deadlock permanently until a server is restarted. When this issue occurs, no new proxies are able to successfully connect to the server. Effectively, the trigger for this deadlock stems from the following return statement: https://github.com/hashicorp/consul/blob/v1.18.0/agent/proxycfg-sources/catalog/config_source.go#L199-L202 When this happens, the entire `syncLoop()` terminates and stops consuming from the following channel: https://github.com/hashicorp/consul/blob/v1.18.0/agent/proxycfg-sources/catalog/config_source.go#L182-L192 Which results in the `ConfigSource.cleanup()` function never receiving a response and holding a mutex indefinitely: https://github.com/hashicorp/consul/blob/v1.18.0/agent/proxycfg-sources/catalog/config_source.go#L241-L247 Because this mutex is shared, it effectively deadlocks the server's ability to process new xDS streams. ---- The fix to this issue involves removing the `chan chan struct{}` used like an RPC-over-channels pattern and replacing it with two distinct channels: + `stopSyncLoopCh` - indicates that the `syncLoop()` should terminate soon. + `syncLoopDoneCh` - indicates that the `syncLoop()` has terminated. Splitting these two concepts out and deferring a `close(syncLoopDoneCh)` in the `syncLoop()` function ensures that the deadlock above should no longer occur. We also now evict xDS connections of all proxies for the corresponding `syncLoop()` whenever it encounters an irrecoverable error. This is done by hoisting the new `syncLoopDoneCh` upwards so that it's visible to the xDS delta processing. Prior to this fix, the behavior was to simply orphan them so they would never receive catalog-registration or service-defaults updates. * Add changelog.	8 months ago
Derek Menteer	eabff257d7	Various bug-fixes and improvements (#20866) ... * Shuffle the list of servers returned by `pbserverdiscovery.WatchServers`. This randomizes the list of servers to help reduce the chance of clients all connecting to the same server simultaneously. Consul-dataplane is one such client that does not randomize its own list of servers. * Fix potential goroutine leak in xDS recv loop. This commit ensures that the goroutine which receives xDS messages from proxies will not block forever if the stream's context is cancelled but the `processDelta()` function never consumes the message (due to being terminated). * Add changelog.	8 months ago
sarahalsmiller	262f435800	NET-6821 Disable Terminating Gateway Auto Host Header Rewrite (#20802) ... * disable terminating gateway auto host rewrite * add changelog * clean up unneeded additional snapshot fields * add new field to docs * squash * fix test	9 months ago
skpratt	57bad0df85	add traffic permissions excludes and tests (#20453) ... * add traffic permissions tests * review fixes * Update internal/mesh/internal/controllers/sidecarproxy/builder/local_app.go Co-authored-by: John Landa <jonathanlanda@gmail.com> --------- Co-authored-by: John Landa <jonathanlanda@gmail.com>	10 months ago
Chris S. Kim	b6f10bc58f	Skip filter chain created by permissive mtls (#20406)	10 months ago
Derek Menteer	3e8ec8d18e	Fix SAN matching on terminating gateways (#20417) ... Fixes issue: hashicorp/consul#20360 A regression was introduced in hashicorp/consul#19954 where the SAN validation matching was reduced from 4 potential types down to just the URI. Terminating gateways will need to match on many fields depending on user configuration, since they make egress calls outside of the cluster. Having more than one matcher behaves like an OR operation, where any match is sufficient to pass the certificate validation. To maintain backwards compatibility with the old untyped `match_subject_alt_names` Envoy behavior, we should match on all 4 enum types. https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/transport_sockets/tls/v3/common.proto#enum-extensions-transport-sockets-tls-v3-subjectaltnamematcher-santype	10 months ago
Matt Keeler	34a32d4ce5	Remove V2 PeerName field from pbresource.Tenancy (#19865) ... The peer name will eventually show up elsewhere in the resource. For now though this rips it out of where we don’t want it to be.	10 months ago
skpratt	0abf8f8426	Net 5092/internal l7 traffic permissions (#20276) ... * wire up L7 Traffic Permissions * testing * update comment	10 months ago
Lord-Y	758ddf84e9	Case sensitive route match (#19647) ... Add case insensitive param on service route match This commit adds in a new feature that allows service routers to specify that paths and path prefixes should ignore upper / lower casing when matching URLs. Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com>	10 months ago
John Murret	7a410d7c5b	NET-6945 - Replace usage of deprecated Envoy field envoy.config.core.v3.HeaderValueOption.append (#20078) ... * NET-6945 - Replace usage of deprecated Envoy field envoy.config.core.v3.HeaderValueOption.append * update proto for v2 and then update xds v2 logic * add changelog * Update 20078.txt to be consistent with existing changelog entries * swap enum values tomatch envoy.	11 months ago
John Murret	d925e4b812	NET-6946 / NET-6941 - Replace usage of deprecated Envoy fields envoy.config.route.v3.HeaderMatcher.safe_regex_match and envoy.type.matcher.v3.RegexMatcher.google_re2 (#20013) ... * NET-6946 - Replace usage of deprecated Envoy field envoy.config.route.v3.HeaderMatcher.safe_regex_match * removing unrelated changes * update golden files * do not set engine type	11 months ago
John Murret	2f335113f8	NET-6943 - Replace usage of deprecated Envoy field envoy.config.router.v3.WeightedCluster.total_weight. (#20011)	11 months ago
John Murret	90cd56c5c3	NET-4774 - replace usage of deprecated Envoy field match_subject_alt_names (#19954)	11 months ago
John Murret	21ea5c92fd	NET-6944 - Replace usage of deprecated Envoy field envoy.extensions.filters.http.lua.v3.Lua.inline_code (#20012)	11 months ago
Nitya Dhanushkodi	9975b8bd73	[NET-5455] Allow disabling request and idle timeouts with negative values in service router and service resolver (#19992) ... * add coverage for testing these timeouts	11 months ago
Derek Menteer	dfab5ade50	Fix ClusterLoadAssignment timeouts dropping endpoints. (#19871) ... When a large number of upstreams are configured on a single envoy proxy, there was a chance that it would timeout when waiting for ClusterLoadAssignments. While this doesn't always immediately cause issues, consul-dataplane instances appear to consistently drop endpoints from their configurations after an xDS connection is re-established (the server dies, random disconnect, etc). This commit adds an `xds_fetch_timeout_ms` config to service registrations so that users can set the value higher for large instances that have many upstreams. The timeout can be disabled by setting a value of `0`. This configuration was introduced to reduce the risk of causing a breaking change for users if there is ever a scenario where endpoints would never be received. Rather than just always blocking indefinitely or for a significantly longer period of time, this config will affect only the service instance associated with it.	12 months ago
Derek Menteer	0ac958f27b	Fix xDS missing endpoint race condition. (#19866) ... This fixes the following race condition: - Send update endpoints - Send update cluster - Recv ACK endpoints - Recv ACK cluster Prior to this fix, it would have resulted in the endpoints NOT existing in Envoy. This occurred because the cluster update implicitly clears the endpoints in Envoy, but we would never re-send the endpoint data to compensate for the loss, because we would incorrectly ACK the invalid old endpoint hash. Since the endpoint's hash did not actually change, they would not be resent. The fix for this is to effectively clear out the invalid pending ACKs for child resources whenever the parent changes. This ensures that we do not store the child's hash as accepted when the race occurs. An escape-hatch environment variable `XDS_PROTOCOL_LEGACY_CHILD_RESEND` was added so that users can revert back to the old legacy behavior in the event that this produces unknown side-effects. Visit the following thread for some extra context on why certainty around these race conditions is difficult: https://github.com/envoyproxy/envoy/issues/13009 This bug report and fix was mostly implemented by @ksmiley with some minor tweaks. Co-authored-by: Keith Smiley <ksmiley@salesforce.com>	12 months ago
Thomas Eckert	8125a32a4e	Add CE version of Gateway Upstream Disambiguation (#19860) ... * Add CE version of gateway-upstream-disambiguation * Use NamespaceOrDefault and PartitionOrDefault * Add Changelog entry * Remove the unneeded reassignment * Use c.ID()	12 months ago
Dhia Ayachi	d93f7f730d	parse config protocol on write to optimize disco-chain compilation (#19829) ... * parse config protocol on write to optimize disco-chain compilation * add changelog	12 months ago
John Murret	780e91688d	Migrate remaining individual resource tests for service mesh to TestAllResourcesFromSnapshot (#19583) ... * migrate expose checks and paths tests to resources_test.go * fix failing expose paths tests * fix the way endpoint resources get created to make expose tests pass. * remove endpoint resources that are already inlined on local_app clusters * renaiming and comments * migrate remaining service mesh tests to resources_test.go * cleanup * update proxystateconverter to skip ading alpn to clusters and listener filterto match v1 behavior	1 year ago
John Murret	f5bf256425	Migrate individual resource tests for API Gateway to TestAllResourcesFromSnapshot (#19584) ... migrate individual api gateway tests to resources_test.go	1 year ago
John Murret	a94fa4c3ed	Migrate individual resource tests for Mesh Gateway to TestAllResourcesFromSnapshot (#19502) ... migrate mesh-gateway tests to resources_test.go	1 year ago
John Murret	4aa95f3d1f	Migrate individual resource tests for Ingress Gateway to TestAllResourcesFromSnapshot (#19506) ... migrate ingress-gateway tests to resources_test.go	1 year ago
John Murret	2553d6e8b9	Migrate individual resource tests for Terminating Gateway to TestAllResourcesFromSnapshot (#19505) ... migrate terminating-gateway tests to resources_test.go	1 year ago
John Murret	7de0b45ba4	Fix xds v2 from creating envoy endpoint resources when already inlined in the cluster (#19580) ... * migrate expose checks and paths tests to resources_test.go * fix failing expose paths tests * fix the way endpoint resources get created to make expose tests pass. * wip * remove endpoint resources that are already inlined on local_app clusters * renaiming and comments	1 year ago
John Murret	5aff19f9bc	Migrate individual resource tests for JWT Provider to TestAllResourcesFromSnapshot (#19511) ... migrate jwt provider tests to resources_test.go	1 year ago
John Murret	903ff7fccb	Migrate individual resource tests for custom configuration to TestAllResourcesFromSnapshot (#19512) ... * Configure TestAllResourcesFromSnapshot to run V2 tests * migrate custom configuration tests to resources_test.go	1 year ago
John Murret	09f73d1abf	Migrate individual resource tests for expose paths and checks to TestAllResourcesFromSnapshot (#19513) ... * migrate expose checks and paths tests to resources_test.go * fix failing expose paths tests	1 year ago
John Murret	7bc2581c81	Migrate individual resource tests for Discovery Chains to TestAllResourcesFromSnapshot (#19508) ... migrate disco chain tests to resources_test.go	1 year ago
John Murret	f115cdb1d5	NET-6385 - Static routes that are inlined in listener filters are also created as a resource. (#19459) ... * cover all protocols in local_app golden tests * fix xds tests * updating latest * fix broken test * add sorting of routers to TestBuildLocalApp to get rid of the flaking * cover all protocols in local_app golden tests * cover all protocols in local_app golden tests * cover all protocols in local_app golden tests * process envoy resource by walking the map. use a map rather than array for envoy resource to prevent duplication. * cleanup. doc strings. * update to latest * fix broken test * update tests after adding sorting of routers in local_app builder tests * do not make endpoints for local_app * fix catalog destinations only by creating clusters for any cluster not already created by walking the graph. * Configure TestAllResourcesFromSnapshot to run V2 tests * wip * fix processing of failover groups * add endpoints and clusters for any clusters that were not created from walking the listener -> path * fix xds v2 golden files for clusters to include failover group clusters	1 year ago
John Murret	74daaa5043	XDS V1 should not make runs for TCP Disco Chains. (#19496) ... * XDS V1 should not make runs for TCP Disco Chains. * update TestEnvoyExtenderWithSnapshot	1 year ago
John Murret	f0cf8f2f40	NET-6294 - v1 Agentless proxycfg datasource errors after v2 changes (#19365)	1 year ago
Michael Zalimeni	a7803bd829	[NET-6305] xds: Ensure v2 route match and protocol are populated for gRPC (#19343) ... * xds: Ensure v2 route match is populated for gRPC Similar to HTTP, ensure that route match config (which is required by Envoy) is populated when default values are used. Because the default matches generated for gRPC contain a single empty `GRPCRouteMatch`, and that proto does not directly support prefix-based config, an interpretation of the empty struct is needed to generate the same output that the `HTTPRouteMatch` is explicitly configured to provide in internal/mesh/internal/controllers/routes/generate.go. * xds: Ensure protocol set for gRPC resources Add explicit protocol in `ProxyStateTemplate` builders and validate it is always set on clusters. This ensures that HTTP filters and `http2_protocol_options` are populated in all the necessary places for gRPC traffic and prevents future unintended omissions of non-TCP protocols. Co-authored-by: John Murret <john.murret@hashicorp.com> --------- Co-authored-by: John Murret <john.murret@hashicorp.com>	1 year ago
Andrew Stucki	e414cbee4a	Use strict DNS for mesh gateways with hostnames (#19268) ... * Use strict DNS for mesh gateways with hostnames * Add changelog	1 year ago
Michael Zalimeni	5e517c5980	[NET-6221] Ensure LB policy set for locality-aware routing (CE) (#19283) ... Ensure LB policy set for locality-aware routing (CE) `overprovisioningFactor` should be overridden with the expected value (100,000) when there are multiple endpoint groups. Update code and tests to enforce this. This is an Enterprise feature. This commit represents the CE portions of the change; tests are added in the corresponding `consul-enterprise` change.	1 year ago
John Maguire	b78465b491	[NET-5810] CE changes for multiple virtual hosts (#19246) ... CE changes for multiple virtual hosts	1 year ago
Thomas Eckert	76c60fdfac	Golden File Tests for TermGW w/ Cluster Peering (#19096) ... Add intention to create golden file for terminating gateway peered trust bundle	1 year ago