371 Commits (21cca2dc5b1b35e1db1151b4dcacefa3975e7d92)

Author	SHA1	Message	Date
Eric Haberkorn	a6d69adcf5	Add default resolvers to disco chains based on the default sameness group (#16837)	2 years ago
Eric Haberkorn	0d1d2fc4c9	add order by locality failover to Consul enterprise (#16791)	2 years ago
Ronald	94ec4eb2f4	copyright headers for agent folder (#16704) ... * copyright headers for agent folder * Ignore test data files * fix proto files and remove headers in agent/uiserver folder * ignore deep-copy files	2 years ago
Derek Menteer	2236975011	Change partition for peers in discovery chain targets (#16769) ... This commit swaps the partition field to the local partition for discovery chains targeting peers. Prior to this change, peer upstreams would always use a value of default regardless of which partition they exist in. This caused several issues in xds / proxycfg because of id mismatches. Some prior fixes were made to deal with one-off id mismatches that this PR also cleans up, since they are no longer needed.	2 years ago
Eric Haberkorn	495ad4c7ef	add enterprise xds tests (#16738)	2 years ago
John Maguire	8dd1d73874	Remove unused are hosts set check (#16691) ... * Remove unused are hosts set check * Remove all traces of unused 'AreHostsSet' parameter * Remove unused Hosts attribute * Remove commented out use of snap.APIGateway.Hosts	2 years ago
Nitya Dhanushkodi	b9bd2c3780	peering: peering partition failover fixes (#16673) ... add local source partition for peered upstreams	2 years ago
John Maguire	1ef9f4dade	Fix route subscription when using namespaces (#16677) ... * Fix route subscription when using namespaces * Update changelog * Fix changelog entry to reference that the bug was enterprise only	2 years ago
Ashvitha	f95ffe0355	Allow HCP metrics collection for Envoy proxies ... Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com> Add a new envoy flag: "envoy_hcp_metrics_bind_socket_dir", a directory where a unix socket will be created with the name `<namespace>_<proxy_id>.sock` to forward Envoy metrics. If set, this will configure: - In bootstrap configuration a local stats_sink and static cluster. These will forward metrics to a loopback listener sent over xDS. - A dynamic listener listening at the socket path that the previously defined static cluster is sending metrics to. - A dynamic cluster that will forward traffic received at this listener to the hcp-metrics-collector service. Reasons for having a static cluster pointing at a dynamic listener: - We want to secure the metrics stream using TLS, but the stats sink can only be defined in bootstrap config. With dynamic listeners/clusters we can use the proxy's leaf certificate issued by the Connect CA, which isn't available at bootstrap time. - We want to intelligently route to the HCP collector. Configuring its addreess at bootstrap time limits our flexibility routing-wise. More on this below. Reasons for defining the collector as an upstream in `proxycfg`: - The HCP collector will be deployed as a mesh service. - Certificate management is taken care of, as mentioned above. - Service discovery and routing logic is automatically taken care of, meaning that no code changes are required in the xds package. - Custom routing rules can be added for the collector using discovery chain config entries. Initially the collector is expected to be deployed to each admin partition, but in the future could be deployed centrally in the default partition. These config entries could even be managed by HCP itself.	2 years ago
R.B. Boyer	9a485cdb49	proxycfg: ensure that an irrecoverable error in proxycfg closes the xds session and triggers a replacement proxycfg watcher (#16497) ... Receiving an "acl not found" error from an RPC in the agent cache and the streaming/event components will cause any request loops to cease under the assumption that they will never work again if the token was destroyed. This prevents log spam (#14144, #9738). Unfortunately due to things like: - authz requests going to stale servers that may not have witnessed the token creation yet - authz requests in a secondary datacenter happening before the tokens get replicated to that datacenter - authz requests from a primary TO a secondary datacenter happening before the tokens get replicated to that datacenter The caller will get an "acl not found" *before* the token exists, rather than just after. The machinery added above in the linked PRs will kick in and prevent the request loop from looping around again once the tokens actually exist. For `consul-dataplane` usages, where xDS is served by the Consul servers rather than the clients ultimately this is not a problem because in that scenario the `agent/proxycfg` machinery is on-demand and launched by a new xDS stream needing data for a specific service in the catalog. If the watching goroutines are terminated it ripples down and terminates the xDS stream, which CDP will eventually re-establish and restart everything. For Consul client usages, the `agent/proxycfg` machinery is ahead-of-time launched at service registration time (called "local" in some of the proxycfg machinery) so when the xDS stream comes in the data is already ready to go. If the watching goroutines terminate it should terminate the xDS stream, but there's no mechanism to re-spawn the watching goroutines. If the xDS stream reconnects it will see no `ConfigSnapshot` and will not get one again until the client agent is restarted, or the service is re-registered with something changed in it. This PR fixes a few things in the machinery: - there was an inadvertent deadlock in fetching snapshot from the proxycfg machinery by xDS, such that when the watching goroutine terminated the snapshots would never be fetched. This caused some of the xDS machinery to get indefinitely paused and not finish the teardown properly. - Every 30s we now attempt to re-insert all locally registered services into the proxycfg machinery. - When services are re-inserted into the proxycfg machinery we special case "dead" ones such that we unilaterally replace them rather that doing that conditionally.	2 years ago
Andrew Stucki	5deffbd95b	Fix issue where terminating gateway service resolvers weren't properly cleaned up (#16498) ... * Fix issue where terminating gateway service resolvers weren't properly cleaned up * Add integration test for cleaning up resolvers * Add changelog entry * Use state test and drop integration test	2 years ago
Andrew Stucki	4b661d1e0c	Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable (#16495) ... * Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable * Regenerate golden files * Add RequestTimeout field * Add changelog entry	2 years ago
Matt Keeler	085c0addc0	Protobuf Refactoring for Multi-Module Cleanliness (#16302) ... Protobuf Refactoring for Multi-Module Cleanliness This commit includes the following: Moves all packages that were within proto/ to proto/private Rewrites imports to account for the packages being moved Adds in buf.work.yaml to enable buf workspaces Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes) Why: In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage. There were some recent changes to have our own ratelimiting annotations. The two combined were not working when I was trying to use them together (attempting to rebase another branch) Buf workspaces should be the solution to the problem Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root. This resulted in proto file name conflicts in the Go global protobuf type registry. The solution to that was to add in a private/ directory into the path within the proto/ directory. That then required rewriting all the imports. Is this safe? AFAICT yes The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc) Other than imports, there were no changes to any generated code as a result of this.	2 years ago
Andrew Stucki	e4a992c581	Fix hostname alignment checks for HTTPRoutes (#16300) ... * Fix hostname alignment checks for HTTPRoutes	2 years ago
Andrew Stucki	b3ddd4d24e	Inline API Gateway TLS cert code (#16295) ... * Include secret type when building resources from config snapshot * First pass at generating envoy secrets from api-gateway snapshot * Update comments for xDS update order * Add secret type + corresponding golden files to existing tests * Initialize test helpers for testing api-gateway resource generation * Generate golden files for new api-gateway xDS resource test * Support ADS for TLS certificates on api-gateway * Configure TLS on api-gateway listeners * Inline TLS cert code * update tests * Add SNI support so we can have multiple certificates * Remove commented out section from helper * regen deep-copy * Add tcp tls test --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>	2 years ago
Thomas Eckert	2460ac99c9	API Gateway Envoy Golden Listener Tests (#16221) ... * Simple API Gateway e2e test for tcp routes * Drop DNSSans since we don't front the Gateway with a leaf cert * WIP listener tests for api-gateway * Return early if no routes * Add back in leaf cert to testing * Fix merge conflicts * Re-add kind to setup * Fix iteration over listener upstreams * New tcp listener test * Add tests for API Gateway with TCP and HTTP routes * Move zero-route check back * Drop generateIngressDNSSANs * Check for chains not routes --------- Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
Derek Menteer	30112288c8	Fix mesh gateways incorrectly matching peer locality. (#16257) ... Fix mesh gateways incorrectly matching peer locality. This fixes an issue where local mesh gateways use an incorrect address when attempting to forward traffic to a peered datacenter. Prior to this change it would use the lan address instead of the wan if the locality matched. This should never be done for peering, since we must route all traffic through the remote mesh gateway.	2 years ago
Derek Menteer	6599a9be1d	Fix nil-pointer panics from proxycfg package. (#16277) ... Prior to this PR, servers / agents would panic and crash if an ingress or api gateway were configured to use a discovery chain that both: 1. Referenced a peered service 2. Had a mesh gateway mode of local This could occur, because code for handling upstream watches was shared between both connect-proxy and the gateways. As a short-term fix, this PR ensures that the maps are always initialized for these gateway services. This PR also wraps the proxycfg execution and service registration calls with recover statements to ensure that future issues like this do not put the server into an unrecoverable state.	2 years ago
Andrew Stucki	9bb0ecfc18	[API Gateway] Add integration test for HTTP routes (#16236) ... * [API Gateway] Add integration test for conflicted TCP listeners * [API Gateway] Update simple test to leverage intentions and multiple listeners * Fix broken unit test * [API Gateway] Add integration test for HTTP routes	2 years ago
Andrew Stucki	8ff2974dbe	[API Gateway] Update simple test to leverage intentions and multiple listeners (#16228) ... * [API Gateway] Add integration test for conflicted TCP listeners * [API Gateway] Update simple test to leverage intentions and multiple listeners * Fix broken unit test * PR suggestions	2 years ago
Andrew Stucki	4c848a554d	Fix missing references to enterprise metadata (#16237)	2 years ago
Andrew Stucki	318ba215ab	[API Gateway] Add integration test for conflicted TCP listeners (#16225)	2 years ago
Andrew Stucki	3b9c569561	Simple API Gateway e2e test for tcp routes (#16222) ... * Simple API Gateway e2e test for tcp routes * Drop DNSSans since we don't front the Gateway with a leaf cert	2 years ago
Thomas Eckert	e81a0c2855	API Gateway to Ingress Gateway Snapshot Translation and Routes to Virtual Routers and Splitters (#16127) ... * Stub proxycfg handler for API gateway * Add Service Kind constants/handling for API Gateway * Begin stubbing for SDS * Add new Secret type to xDS order of operations * Continue stubbing of SDS * Iterate on proxycfg handler for API gateway * Handle BoundAPIGateway config entry subscription in proxycfg-glue * Add API gateway to config snapshot validation * Add API gateway to config snapshot clone, leaf, etc. * Subscribe to bound route + cert config entries on bound-api-gateway * Track routes + certs on API gateway config snapshot * Generate DeepCopy() for types used in watch.Map * Watch all active references on api-gateway, unwatch inactive * Track loading of initial bound-api-gateway config entry * Use proper proto package for SDS mapping * Use ResourceReference instead of ServiceName, collect resources * Fix typo, add + remove TODOs * Watch discovery chains for TCPRoute * Add TODO for updating gateway services for api-gateway * make proto * Regenerate deep-copy for proxycfg * Set datacenter on upstream ID from query source * Watch discovery chains for http-route service backends * Add ServiceName getter to HTTP+TCP Service structs * Clean up unwatched discovery chains on API Gateway * Implement watch for ingress leaf certificate * Collect upstreams on http-route + tcp-route updates * Remove unused GatewayServices update handler * Remove unnecessary gateway services logic for API Gateway * Remove outdate TODO * Use .ToIngress where appropriate, including TODO for cleaning up * Cancel before returning error * Remove GatewayServices subscription * Add godoc for handlerAPIGateway functions * Update terminology from Connect => Consul Service Mesh Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690 * Add missing TODO * Remove duplicate switch case * Rerun deep-copy generator * Use correct property on config snapshot * Remove unnecessary leaf cert watch * Clean up based on code review feedback * Note handler properties that are initialized but set elsewhere * Add TODO for moving helper func into structs pkg * Update generated DeepCopy code * gofmt * Begin stubbing for SDS * Start adding tests * Remove second BoundAPIGateway case in glue * TO BE PICKED: fix formatting of str * WIP * Fix merge conflict * Implement HTTP Route to Discovery Chain config entries * Stub out function to create discovery chain * Add discovery chain merging code (#16131) * Test adding TCP and HTTP routes * Add some tests for the synthesizer * Run go mod tidy * Pairing with N8 * Run deep copy * Clean up GatewayChainSynthesizer * Fix missing assignment of BoundAPIGateway topic * Separate out synthesizeChains and toIngressTLS * Fix build errors * Ensure synthesizer skips non-matching routes by protocol * Rebase on N8s work * Generate DeepCopy() for API gateway listener types * Improve variable name * Regenerate DeepCopy() code * Fix linting issue * fix protobuf import * Fix more merge conflict errors * Fix synthesize test * Run deep copy * Add URLRewrite to proto * Update agent/consul/discoverychain/gateway_tcproute.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Remove APIGatewayConfigEntry that was extra * Error out if route kind is unknown * Fix formatting errors in proto --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
Nathan Coleman	72a73661c9	Implement APIGateway proxycfg snapshot (#16194) ... * Stub proxycfg handler for API gateway * Add Service Kind constants/handling for API Gateway * Begin stubbing for SDS * Add new Secret type to xDS order of operations * Continue stubbing of SDS * Iterate on proxycfg handler for API gateway * Handle BoundAPIGateway config entry subscription in proxycfg-glue * Add API gateway to config snapshot validation * Add API gateway to config snapshot clone, leaf, etc. * Subscribe to bound route + cert config entries on bound-api-gateway * Track routes + certs on API gateway config snapshot * Generate DeepCopy() for types used in watch.Map * Watch all active references on api-gateway, unwatch inactive * Track loading of initial bound-api-gateway config entry * Use proper proto package for SDS mapping * Use ResourceReference instead of ServiceName, collect resources * Fix typo, add + remove TODOs * Watch discovery chains for TCPRoute * Add TODO for updating gateway services for api-gateway * make proto * Regenerate deep-copy for proxycfg * Set datacenter on upstream ID from query source * Watch discovery chains for http-route service backends * Add ServiceName getter to HTTP+TCP Service structs * Clean up unwatched discovery chains on API Gateway * Implement watch for ingress leaf certificate * Collect upstreams on http-route + tcp-route updates * Remove unused GatewayServices update handler * Remove unnecessary gateway services logic for API Gateway * Remove outdate TODO * Use .ToIngress where appropriate, including TODO for cleaning up * Cancel before returning error * Remove GatewayServices subscription * Add godoc for handlerAPIGateway functions * Update terminology from Connect => Consul Service Mesh Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690 * Add missing TODO * Remove duplicate switch case * Rerun deep-copy generator * Use correct property on config snapshot * Remove unnecessary leaf cert watch * Clean up based on code review feedback * Note handler properties that are initialized but set elsewhere * Add TODO for moving helper func into structs pkg * Update generated DeepCopy code * gofmt * Generate DeepCopy() for API gateway listener types * Improve variable name * Regenerate DeepCopy() code * Fix linting issue * Temporarily remove the secret type from resource generation	2 years ago
Derek Menteer	1b02749375	Add extension validation on config save and refactor extensions. (#16110)	2 years ago
Nitya Dhanushkodi	8728a4496c	troubleshoot: service to service validation (#16096) ... * Add Tproxy support to Envoy Extensions (this is needed for service to service validation) * Add validation for Envoy configuration for an upstream service * Use both /config_dump and /cluster to validate Envoy configuration This is because of a bug in Envoy where the EndpointsConfigDump does not include a cluster_name, making it impossible to match an endpoint to verify it exists. This removes endpoints support for builtin extensions since only the validate plugin was using it, and it is no longer used. It also removes test cases for endpoint validation. Endpoints validation now only occurs in the top level test from config_dump and clusters json files. Co-authored-by: Eric <eric@haberkorn.co>	2 years ago
Michael Wilkerson	a1498b015d	Mw/lambda envoy extension parse region (#4107) (#16069) ... * updated builtin extension to parse region directly from ARN - added a unit test - added some comments/light refactoring * updated golden files with proper ARNs - ARNs need to be right format now that they are being processed * updated tests and integration tests - removed 'region' from all EnvoyExtension arguments - added properly formatted ARN which includes the same region found in the removed "Region" field: 'us-east-1'	2 years ago
Derek Menteer	2facf50923	Fix configuration merging for implicit tproxy upstreams. (#16000) ... Fix configuration merging for implicit tproxy upstreams. Change the merging logic so that the wildcard upstream has correct proxy-defaults and service-defaults values combined into it. It did not previously merge all fields, and the wildcard upstream did not exist unless service-defaults existed (it ignored proxy-defaults, essentially). Change the way we fetch upstream configuration in the xDS layer so that it falls back to the wildcard when no matching upstream is found. This is what allows implicit peer upstreams to have the correct "merged" config. Change proxycfg to always watch local mesh gateway endpoints whenever a peer upstream is found. This simplifies the logic so that we do not have to inspect the "merged" configuration on peer upstreams to extract the mesh gateway mode.	2 years ago
Derek Menteer	1f7e7abeac	Fix issue with incorrect proxycfg watch on upstream peer-targets. (#15865) ... This fixes an issue where the incorrect partition was given to the upstream target watch, which meant that failover logic would not work correctly.	2 years ago
Nitya Dhanushkodi	c7ef04c597	[OSS] extensions: refactor PluginConfiguration into a more generic type ExtensionConfiguration (#15846) ... * extensions: refactor PluginConfiguration into a more generic type ExtensionConfiguration Also: * adds endpoints configuration to lambda golden tests * uses string constant for builtin/aws/lambda Co-authored-by: Eric <eric@haberkorn.co>	2 years ago
Nitya Dhanushkodi	d382ca0aec	extensions: refactor serverless plugin to use extensions from config entry fields (#15817) ... docs: update config entry docs and the Lambda manual registration docs Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Eric <eric@haberkorn.co>	2 years ago
cskh	04bf24c8c1	feat(ingress-gateway): support outlier detection of upstream service for ingress gateway (#15614) ... * feat(ingress-gateway): support outlier detection of upstream service for ingress gateway * changelog Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>	2 years ago
Derek Menteer	e87d35e313	Fix DialedDirectly configuration for Consul dataplane. (#15760) ... Fix DialedDirectly configuration for Consul dataplane.	2 years ago
Derek Menteer	97ec5279aa	Fix local mesh gateway with peering discovery chains. (#15690) ... Fix local mesh gateway with peering discovery chains. Prior to this patch, discovery chains with peers would not properly honor the mesh gateway mode for two reasons. 1. An incorrect target upstream ID was used to lookup the mesh gateway mode. To fix this, the parent upstream uid is now used instead of the discovery-chain-target-uid to find the intended mesh gateway mode. 2. The watch for local mesh gateways was never initialized for discovery chains. To fix this, the discovery chains are now scanned, and a local GW watch is spawned if: the mesh gateway mode is local and the target is a peering connection.	2 years ago
James Oulman	7e78fb7818	Add support for configuring Envoys route idle_timeout (#14340) ... * Add idleTimeout Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
Kyle Schochenmaier	bf0f61a878	removes ioutil usage everywhere which was deprecated in go1.16 (#15297) ... * update go version to 1.18 for api and sdk, go mod tidy * removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward. Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	2 years ago
Eric Haberkorn	c340922991	Log Warnings When Peering With Mesh Gateway Mode None (#15304) ... warn when mesh gateway mode is set to none for peering	2 years ago
Derek Menteer	418bd62c44	Fix mesh gateway configuration with proxy-defaults (#15186) ... * Fix mesh gateway proxy-defaults not affecting upstreams. * Clarify distinction with upstream settings Top-level mesh gateway mode in proxy-defaults and service-defaults gets merged into NodeService.Proxy.MeshGateway, and only gets merged with the mode attached to an an upstream in proxycfg/xds. * Fix mgw mode usage for peered upstreams There were a couple issues with how mgw mode was being handled for peered upstreams. For starters, mesh gateway mode from proxy-defaults and the top-level of service-defaults gets stored in NodeService.Proxy.MeshGateway, but the upstream watch for peered data was only considering the mesh gateway config attached in NodeService.Proxy.Upstreams[i]. This means that applying a mesh gateway mode via global proxy-defaults or service-defaults on the downstream would not have an effect. Separately, transparent proxy watches for peered upstreams didn't consider mesh gateway mode at all. This commit addresses the first issue by ensuring that we overlay the upstream config for peered upstreams as we do for non-peered. The second issue is addressed by re-using setupWatchesForPeeredUpstream when handling transparent proxy updates. Note that for transparent proxies we do not yet support mesh gateway mode per upstream, so the NodeService.Proxy.MeshGateway mode is used. * Fix upstream mesh gateway mode handling in xds This commit ensures that when determining the mesh gateway mode for peered upstreams we consider the NodeService.Proxy.MeshGateway config as a baseline. In absense of this change, setting a mesh gateway mode via proxy-defaults or the top-level of service-defaults will not have an effect for peered upstreams. * Merge service/proxy defaults in cfg resolver Previously the mesh gateway mode for connect proxies would be merged at three points: 1. On servers, in ComputeResolvedServiceConfig. 2. On clients, in MergeServiceConfig. 3. On clients, in proxycfg/xds. The first merge returns a ServiceConfigResponse where there is a top-level MeshGateway config from proxy/service-defaults, along with per-upstream config. The second merge combines per-upstream config specified at the service instance with per-upstream config specified centrally. The third merge combines the NodeService.Proxy.MeshGateway config containing proxy/service-defaults data with the per-upstream mode. This third merge is easy to miss, which led to peered upstreams not considering the mesh gateway mode from proxy-defaults. This commit removes the third merge, and ensures that all mesh gateway config is available at the upstream. This way proxycfg/xds do not need to do additional overlays. * Ensure that proxy-defaults is considered in wc Upstream defaults become a synthetic Upstream definition under a wildcard key "*". Now that proxycfg/xds expect Upstream definitions to have the final MeshGateway values, this commit ensures that values from proxy-defaults/service-defaults are the default for this synthetic upstream. * Add changelog. Co-authored-by: freddygv <freddy@hashicorp.com>	2 years ago
Derek Menteer	b64972d486	Bring back parameter ServerExternalAddresses in GenerateToken endpoint (#15267) ... Re-add ServerExternalAddresses parameter in GenerateToken endpoint This reverts commit 5e156772f6 and adds extra functionality to support newer peering behaviors.	2 years ago
cskh	a3f57cc5e8	fix(mesh-gateway): remove deregistered service from mesh gateway (#15272) ... * fix(mesh-gateway): remove deregistered service from mesh gateway * changelog Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com> Co-authored-by: Evan Culver <eculver@users.noreply.github.com>	2 years ago
Derek Menteer	c064ddf606	Backport test fix from ent. (#15279)	2 years ago
Eric Haberkorn	1804b58799	Fix a bug in mesh gateway proxycfg where ACL tokens aren't passed. (#15273)	2 years ago
Luke Kysow	d3aa2bd9c5	ingress-gateways: don't log error when registering gateway (#15001) ... * ingress-gateways: don't log error when registering gateway Previously, when an ingress gateway was registered without a corresponding ingress gateway config entry, an error was logged because the watch on the config entry returned a nil result. This is expected so don't log an error.	2 years ago
Chris S. Kim	bde57c0dd0	Regenerate files according to 1.19.2 formatter	2 years ago
Dan Upton	f8b4b41205	proxycfg: fix goroutine leak when service is re-registered (#14988) ... Fixes a bug where we'd leak a goroutine in state.run when the given context was canceled while there was a pending update.	2 years ago
Freddy	24d0c8801a	Merge pull request #14981 from hashicorp/peering/dial-through-gateways	2 years ago
Dan Upton	328e3ff563	proxycfg: rate-limit delivery of config snapshots (#14960) ... Adds a user-configurable rate limiter to proxycfg snapshot delivery, with a default limit of 250 updates per second. This addresses a problem observed in our load testing of Consul Dataplane where updating a "global" resource such as a wildcard intention or the proxy-defaults config entry could starve the Raft or Memberlist goroutines of CPU time, causing general cluster instability.	2 years ago
Dan Upton	e6b55d1d81	perf: remove expensive reflection from xDS hot path (#14934) ... Replaces the reflection-based implementation of proxycfg's ConfigSnapshot.Clone with code generated by deep-copy. While load testing server-based xDS (for consul-dataplane) we discovered this method is extremely expensive. The ConfigSnapshot struct, directly or indirectly, contains a copy of many of the structs in the agent/structs package, which creates a large graph for copystructure.Copy to traverse at runtime, on every proxy reconfiguration.	2 years ago
freddygv	c77123a2aa	Use split var in tests	2 years ago
freddygv	bf51021c07	Use split wildcard partition name ... This way OSS avoids passing a non-empty label, which will be rejected in OSS consul.	2 years ago
Riddhi Shah	345191a0df	Service http checks data source for agentless proxies (#14924) ... Adds another datasource for proxycfg.HTTPChecks, for use on server agents. Typically these checks are performed by local client agents and there is no equivalent of this in agentless (where servers configure consul-dataplane proxies). Hence, the data source is mostly a no-op on servers but in the case where the service is present within the local state, it delegates to the cache data source.	2 years ago
James Oulman	b8bd7a3058	Configure Envoy alpn_protocols based on service protocol (#14356) ... * Configure Envoy alpn_protocols based on service protocol * define alpnProtocols in a more standard way * http2 protocol should be h2 only * formatting * add test for getAlpnProtocol() * create changelog entry * change scope is connect-proxy * ignore errors on ParseProxyConfig; fixes linter * add tests for grpc and http2 public listeners * remove newlines from PR * Add alpn_protocol configuration for ingress gateway * Guard against nil tlsContext * add ingress gateway w/ TLS tests for gRPC and HTTP2 * getAlpnProtocols: add TCP protocol test * add tests for ingress gateway with grpc/http2 and per-listener TLS config * add tests for ingress gateway with grpc/http2 and per-listener TLS config * add Gateway level TLS config with mixed protocol listeners to validate ALPN * update changelog to include ingress-gateway * add http/1.1 to http2 ALPN * go fmt * fix test on custom-trace-listener	2 years ago
freddygv	5f97223822	Simplify mgw watch mgmt	2 years ago
DanStough	77ab28c5c7	feat: xDS updates for peerings control plane through mesh gw	2 years ago
Eric Haberkorn	1633cf20ea	Make the mesh gateway changes to allow `local` mode for cluster peering data plane traffic (#14817) ... Make the mesh gateway changes to allow `local` mode for cluster peering data plane traffic	2 years ago
Derek Menteer	a279d2d329	Fix explicit tproxy listeners with discovery chains. (#14751) ... Fix explicit tproxy listeners with discovery chains.	2 years ago
Alex Oskotsky	13da2c5fad	Add the ability to retry on reset connection to service-routers (#12890)	2 years ago
Freddy	d9fe3578ac	Merge pull request #14734 from hashicorp/NET-643-update-mesh-gateway-envoy-config-for-inbound-peering-control-plane-traffic	2 years ago
freddygv	b15d41534f	Update xds generation for peering over mesh gws ... This commit adds the xDS resources needed for INBOUND traffic from peer clusters: - 1 filter chain for all inbound peering requests. - 1 cluster for all inbound peering requests. - 1 endpoint per voting server with the gRPC TLS port configured. There is one filter chain and cluster because unlike with WAN federation, peer clusters will not attempt to dial individual servers. Peer clusters will only dial the local mesh gateway addresses.	2 years ago
Ashwin Venkatesh	4ba260958c	bug: watch local mesh gateways in non-default partitions with agentless (#14799)	2 years ago
cskh	69f40df548	feat(ingress gateway: support configuring limits in ingress-gateway c… (#14749) ... * feat(ingress gateway: support configuring limits in ingress-gateway config entry - a new Defaults field with max_connections, max_pending_connections, max_requests is added to ingress gateway config entry - new field max_connections, max_pending_connections, max_requests in individual services to overwrite the value in Default - added unit test and integration test - updated doc Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	2 years ago
Eric Haberkorn	6570d5f004	Enable outbound peered requests to go through local mesh gateway (#14763)	2 years ago
Nick Ethier	1c1b0994b8	add HCP integration component (#14723) ... * add HCP integration * lint: use non-deprecated logging interface	2 years ago
freddygv	d818d7b096	Manage local server watches depending on mesh cfg ... Routing peering control plane traffic through mesh gateways can be enabled or disabled at runtime with the mesh config entry. This commit updates proxycfg to add or cancel watches for local servers depending on this central config. Note that WAN federation over mesh gateways is determined by a service metadata flag, and any updates to the gateway service registration will force the creation of a new snapshot. If enabled, WAN-fed over mesh gateways will trigger a local server watch on initialize(). Because of this we will only add/remove server watches if WAN federation over mesh gateways is disabled.	2 years ago
Eric Haberkorn	aa8268e50c	Implement Cluster Peering Redirects (#14445) ... implement cluster peering redirects	2 years ago
Daniel Upton	8c46e48e0d	proxycfg-glue: server-local implementation of IntentionUpstreamsDestination ... This is the OSS portion of enterprise PR 2463. Generalises the serverIntentionUpstreams type to support matching on a service or destination.	2 years ago
Daniel Upton	f8dba7e9ac	proxycfg-glue: server-local implementation of InternalServiceDump ... This is the OSS portion of enterprise PR 2489. This PR introduces a server-local implementation of the proxycfg.InternalServiceDump interface that sources data from a blocking query against the server's state store. For simplicity, it only implements the subset of the Internal.ServiceDump RPC handler actually used by proxycfg - as such the result type has been changed to IndexedCheckServiceNodes to avoid confusion.	2 years ago
Eric Haberkorn	3726a0ab7a	Finish up cluster peering failover (#14396)	2 years ago
Daniel Upton	13c04a13af	proxycfg: terminate stream on irrecoverable errors ... This is the OSS portion of enterprise PR 2339. It improves our handling of "irrecoverable" errors in proxycfg data sources. The canonical example of this is what happens when the ACL token presented by Envoy is deleted/revoked. Previously, the stream would get "stuck" until the xDS server re-checked the token (after 5 minutes) and terminated the stream. Materializers would also sit burning resources retrying something that could never succeed. Now, it is possible for data sources to mark errors as "terminal" which causes the xDS stream to be closed immediately. Similarly, the submatview.Store will evict materializers when it observes they have encountered such an error.	2 years ago
Eric Haberkorn	58901ad7df	Cluster peering failover disco chain changes (#14296)	2 years ago
Dhia Ayachi	6fd65a4a45	Tgtwy egress HTTP support (#13953) ... * add golden files * add support to http in tgateway egress destination * fix slice sorting to include both address and port when using server_names * fix listener loop for http destination * fix routes to generate a route per port and a virtualhost per port-address combination * sort virtual hosts list to have a stable order * extract redundant serviceNode	2 years ago
Dhia Ayachi	256694b603	inject gateway addons to destination clusters (#13951)	2 years ago
Kyle Havlovitz	93de25f87c	Merge pull request #13872 from hashicorp/remove-upstream-log ... Remove extra logging from ingress upstream watch shutdown	2 years ago
DanStough	2da8949d78	feat: convert destination address to slice	2 years ago
freddygv	b544ce6485	Add ACL enforcement to peering endpoints	2 years ago
Kyle Havlovitz	016f963e7e	Remove excess debug log from ingress upstream shutdown	2 years ago
Kyle Havlovitz	0be7d923dc	Cancel upstream watches when the discovery chain has been removed	2 years ago
Kyle Havlovitz	31318d7049	Fix duplicate Notify calls for discovery chains in ingress gateways	2 years ago
Chris S. Kim	495936300e	Make envoy resources for inferred peered upstreams (#13758) ... Peered upstreams has a separate loop in xds from discovery chain upstreams. This PR adds similar but slightly modified code to add filters for peered upstream listeners, clusters, and endpoints in the case of transparent proxy.	2 years ago
Dan Stough	49f3dadb8f	feat: connect proxy xDS for destinations ... Signed-off-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
Chris S. Kim	f56810132f	Check if an upstream is implicit from either intentions or peered services	2 years ago
Chris S. Kim	02cff2394d	Use new maps for proxycfg peered data	2 years ago
Chris S. Kim	7f32cba735	Add new watch.Map type to refactor proxycfg	2 years ago
Kyle Havlovitz	9097e2b0f0	Merge pull request #13699 from hashicorp/tgate-http2-upstream ... Respect http2 protocol for upstreams of terminating gateways	2 years ago
Kyle Havlovitz	7d0c692374	Use protocol from resolved config entry, not gateway service	2 years ago
R.B. Boyer	2317f37b4d	state: prohibit exported discovery chains to have cross-datacenter or cross-partition references (#13726) ... Because peerings are pairwise, between two tuples of (datacenter, partition) having any exported reference via a discovery chain that crosses out of the peered datacenter or partition will ultimately not be able to work for various reasons. The biggest one is that there is no way in the ultimate destination to configure an intention that can allow an external SpiffeID to access a service. This PR ensures that a user simply cannot do this, so they won't run into weird situations like this.	2 years ago
Kyle Havlovitz	439eccdd80	Respect http2 protocol for upstreams of terminating gateways	2 years ago
Daniel Upton	37ccbd2826	proxycfg: server-local intentions data source ... This is the OSS portion of enterprise PR 2141. This commit provides a server-local implementation of the `proxycfg.Intentions` interface that sources data from streaming events. It adds events for the `service-intentions` config entry type, and then consumes event streams (via materialized views) for the service's explicit intentions and any applicable wildcard intentions, merging them into a single list of intentions. An alternative approach I considered was to consume _all_ intention events (via `SubjectWildcard`) and filter out the irrelevant ones. This would admittedly remove some complexity in the `agent/proxycfg-glue` package but at the expense of considerable overhead from waking potentially many thousands of connect proxies every time any intention is updated.	2 years ago
Chris S. Kim	d8b7940e40	Add internal endpoint to fetch peered upstream candidates from VirtualIP table (#13642) ... For initial cluster peering TProxy support we consider all imported services of a partition to be potential upstreams. We leverage the VirtualIP table because it stores plain service names (e.g. "api", not "api-sidecar-proxy").	2 years ago
R.B. Boyer	31b95c747b	xds: modify rbac rules to use the XFCC header for peered L7 enforcement (#13629) ... When the protocol is http-like, and an intention has a peered source then the normal RBAC mTLS SAN field check is replaces with a joint combo of: mTLS SAN field must be the service's local mesh gateway leaf cert AND the first XFCC header (from the MGW) must have a URI field that matches the original intention source Also: - Update the regex program limit to be much higher than the teeny defaults, since the RBAC regex constructions are more complicated now. - Fix a few stray panics in xds generation.	2 years ago
R.B. Boyer	1a9c86ea8f	xds: mesh gateways now correctly load up peer-exported discovery chains using L7 protocols (#13624) ... A mesh gateway will now configure the filter chains for L7 exported services using the correct discovery chain information.	2 years ago
Chris S. Kim	fb5eb20563	Pass trust domain to RBAC to validate and fix use of wrong peer trust bundles (#13508)	2 years ago
DanStough	4b402e3119	feat: tgtwy xDS generation for destinations ... Signed-off-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
R.B. Boyer	201d1458c3	xds: mesh gateways now have their own leaf certificate when involved in a peering (#13460) ... This is only configured in xDS when a service with an L7 protocol is exported. They also load any relevant trust bundles for the peered services to eventually use for L7 SPIFFE validation during mTLS termination.	2 years ago
R.B. Boyer	f557509e58	xds: allow for peered upstreams to use tagged addresses that are hostnames (#13422) ... Mesh gateways can use hostnames in their tagged addresses (#7999). This is useful if you were to expose a mesh gateway using a cloud networking load balancer appliance that gives you a DNS name but no reliable static IPs. Envoy cannot accept hostnames via EDS and those must be configured using CDS. There was already logic when configuring gateways in other locations in the code, but given the illusions in play for peering the downstream of a peered service wasn't aware that it should be doing that. Also: - ensuring that we always try to use wan-like addresses to cross peer boundaries.	3 years ago
R.B. Boyer	ab758b7b32	peering: allow mesh gateways to proxy L4 peered traffic (#13339) ... Mesh gateways will now enable tcp connections with SNI names including peering information so that those connections may be proxied. Note: this does not change the callers to use these mesh gateways.	3 years ago
Dan Upton	b168424398	xds: remove HTTPCheckFetcher dependency (#13366) ... This is the OSS portion of enterprise PR 1994 Rather than directly interrogating the agent-local state for HTTP checks using the `HTTPCheckFetcher` interface, we now rely on the config snapshot containing the checks. This reduces the number of changes required to support server xDS sessions. It's not clear why the fetching approach was introduced in 931d167ebb.	3 years ago
R.B. Boyer	019aeaa57d	peering: update how cross-peer upstreams and represented in proxycfg and rendered in xds (#13362) ... This removes unnecessary, vestigal remnants of discovery chains.	3 years ago
Freddy	a09c776645	Update public listener with SPIFFE Validator ... Envoy's SPIFFE certificate validation extension allows for us to validate against different root certificates depending on the trust domain of the dialing proxy. If there are any trust bundles from peers in the config snapshot then we use the SPIFFE validator as the validation context, rather than the usual TrustedCA. The injected validation config includes the local root certificates as well.	3 years ago