164 Commits (b757624b5941517e28e5425e460dfed81239702f)

Author	SHA1	Message	Date
Paul Banks	ab27214a10	Minor improvements to SDS server from review	3 years ago
Paul Banks	2b755d7b3f	Allow skipping v2 compat tests for SDS as it's only the SDS server integration that doesn't support v2	3 years ago
Paul Banks	e01c3585a5	Fix integration tests in CI - serve SDS certs from the Docker image not a mounted path	3 years ago
Paul Banks	843079f33a	Fix integration test for older Envoy versions	3 years ago
Paul Banks	cd8ad007fe	Add basic integration test for Envoy ingress with SDS	3 years ago
Evan Culver	7605dff46e	add envoy 1.19.1	3 years ago
Paul Banks	81eb706906	Add Envoy integration test to show Header manip can interpolate Envoy variables	3 years ago
Paul Banks	1b9632531a	Integration tests for all new header manip features	3 years ago
Freddy	8d83d27674	connect: update envoy supported versions to latest patch release ... (#10961) Relevant advisory: https://github.com/envoyproxy/envoy/security/advisories/GHSA-6g4j-5vrw-2m8h	3 years ago
Kyle Havlovitz	073b6c8411	oss: Rename default partition	3 years ago
Matt Keeler	caafc02449	hcs-1936: Prepare for adding license auto-retrieval to auto-config in enterprise	4 years ago
R.B. Boyer	3b50a55533	connect: update supported envoy versions to 1.18.3, 1.17.3, 1.16.4, and 1.15.5 (#10231)	4 years ago
Daniel Nephin	426565b68c	fix failing integration tests ... The new IDs include a leading slash for the partition ID section	4 years ago
R.B. Boyer	abc1dc0fe9	connect: update supported envoy versions to 1.18.2, 1.17.2, 1.16.3, and 1.15.4 (#10101) ... The only thing that needed fixing up pertained to this section of the 1.18.x release notes: > grpc_stats: the default value for stats_for_all_methods is switched from true to false, in order to avoid possible memory exhaustion due to an untrusted downstream sending a large number of unique method names. The previous default value was deprecated in version 1.14.0. This only changes the behavior when the value is not set. The previous behavior can be used by setting the value to true. This behavior change by be overridden by setting runtime feature envoy.deprecated_features.grpc_stats_filter_enable_stats_for_all_methods_by_default. For now to maintain status-quo I'm explicitly setting `stats_for_all_methods=true` in all versions to avoid relying upon the default. Additionally the naming of the emitted metrics for these gRPC requests changed slightly so the integration test assertions for `case-grpc` needed adjusting.	4 years ago
R.B. Boyer	71d45a3460	Support Incremental xDS mode (#9855) ... This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged. Union of all commit messages follows to give an overarching summary: xds: exclusively support incremental xDS when using xDS v3 Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail. Work around a strange older envoy behavior involving empty CDS responses over incremental xDS. xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead. Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client. xds: pull out checkStreamACLs method in advance of a later commit xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty. This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10. xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time	4 years ago
R.B. Boyer	52205ac201	test: switch envoy integration tests to use pkill instead of ps+grep+awk+kill (#10097)	4 years ago
Freddy	a11ea6254e	Check for optionally prepended namespace in upstream assertion (#10049)	4 years ago
Yong Wen Chua	516335e415	Update assertion to not check for port	4 years ago
Yong Wen Chua	409768d6e5	Merge branch 'master' of github.com:hashicorp/consul into tg-rewrite	4 years ago
R.B. Boyer	398b766532	xds: default to speaking xDS v3, but allow for v2 to be spoken upon request (#9658) ... - Also add support for envoy 1.17.0	4 years ago
Yong Wen Chua	ec8fecbf61	Add integration test check	4 years ago
R.B. Boyer	3b6ffc447b	xds: remove deprecated usages of xDS (#9602) ... Note that this does NOT upgrade to xDS v3. That will come in a future PR. Additionally: - Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated. - Shuffled some agent/xds imports in advance of a later xDS v3 upgrade. - Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR. Fixes #8425	4 years ago
R.B. Boyer	39effd620c	xds: only try to create an ipv6 expose checks listener if ipv6 is supported by the kernel (#9765) ... Fixes #9311 This only fails if the kernel has ipv6 hard-disabled. It is not sufficient to merely not provide an ipv6 address for a network interface.	4 years ago
Alvin Huang	bf5b76851a	remove reference to docker/ path for old docker mirror (#9783)	4 years ago
Michele Degges	fee295b685	Remove jfrog references (#9782)	4 years ago
R.B. Boyer	6eeccc93ce	connect: update supported envoy point releases to 1.16.2, 1.15.3, 1.14.6, 1.13.7 (#9737)	4 years ago
Daniel Nephin	79284b41ea	Pin alpine/socat image to a version. ... To fix failing integration tests. The latest version (`1.7.4.0-r0`) appears to not be catting all the bytes, so the expected metrics are missing in the output.	4 years ago
Freddy	fe728855ed	Add DC and NS support for Envoy metrics (#9207) ... This PR updates the tags that we generate for Envoy stats. Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.	4 years ago
R.B. Boyer	5afd04897c	test: use direct service registration in envoy integration tests (#9138) ... This has the biggest impact on enterprise test cases that use namespaced registrations, which prior to this change sometimes failed the initial registration because the namespace was not yet created.	4 years ago
R.B. Boyer	8baf158ea8	Revert "Add namespace support for metrics (OSS) (#9117)" (#9124) ... This reverts commit 06b3b017d3.	4 years ago
Freddy	06b3b017d3	Add namespace support for metrics (OSS) (#9117)	4 years ago
R.B. Boyer	44bd8bd2ae	use the docker proxy for more envoy integration test containers (#9085)	4 years ago
R.B. Boyer	957293d884	wait_for_namespace should take two args (#9086)	4 years ago
Alvin Huang	ae6185a554	use hashicorp docker mirror in envoy helper (#9080)	4 years ago
R.B. Boyer	d7b37e3c6e	fix envoy integ test wait_for_namespace to actually work on CI (#9082)	4 years ago
Alvin Huang	ea88956c9c	use hashicorp docker mirror to prevent rate limit (#9070)	4 years ago
R.B. Boyer	a66c4591d7	agent: introduce path allow list for requests going through the metrics proxy (#9059) ... Added a new option `ui_config.metrics_proxy.path_allowlist`. This defaults to `["/api/v1/query", "/api/v1/query_range"]` when the metrics provider is set to `prometheus`. Requests that do not use one of the allow-listed paths (via exact match) get a 403 Forbidden response instead.	4 years ago
R.B. Boyer	b724e096c2	add namespace waiting function to envoy integration tests (#9051)	4 years ago
R.B. Boyer	8e70c3fc25	missed adding the test delay to the l7-intentions envoy integration test (#9052)	4 years ago
R.B. Boyer	d7c7858d87	Fix even more test flakes in intentions related envoy integration tests (#9013) ... The key thing here is to use `curl --no-keepalive` so that envoy pre-1.15 tests will reliably use the latest listener every time. Extra: - Switched away from editing line-item intentions the legacy way. - Removed some teardown scripts, as we don't share anything between cases anyway - Removed unnecessary use of `run` in some places.	4 years ago
R.B. Boyer	934c65ad77	fix flaky envoy integration tests involving intentions (#8996) ... There is a delay between an intentions change being made, and it being reflected in the Envoy runtime configuration. Now that the enforcement happens inside of Envoy instead of over in the agent, our tests need to explicitly wait until the xDS reconfiguration is complete before attempting to assert intentions worked. Also remove a few double retry loops.	4 years ago
R.B. Boyer	a2c50d3303	connect: add support for envoy 1.16.0, drop support for 1.12.x, and bump point releases as well (#8944) ... Supported versions will be: "1.16.0", "1.15.2", "1.14.5", "1.13.6"	4 years ago
R.B. Boyer	0bf62246e5	speed up envoy integration tests by removing docker-compose (#8982) ... This speeds up individual envoy integration test runs from ~23m to ~14m. It's also a pre-req for possibly switching to doing the tests entirely within Go (no shell-outs).	4 years ago
R.B. Boyer	da4ec9ff27	restore the discovery of tests cases by file system existence (#8983)	4 years ago
R.B. Boyer	a68c0ce1e7	speed up envoy integ tests by not politely stopping containers before destroying them (#8969) ... In local testing this sped up the stop_services call from 11s to 1s per test.	4 years ago
R.B. Boyer	1b413b0444	connect: support defining intentions using layer 7 criteria (#8839) ... Extend Consul’s intentions model to allow for request-based access control enforcement for HTTP-like protocols in addition to the existing connection-based enforcement for unspecified protocols (e.g. tcp).	4 years ago
R.B. Boyer	9801ef8eb1	agent: enable enable_central_service_config by default (#8746)	4 years ago
Jack	9e1c6727f9	Add http2 and grpc support to ingress gateways (#8458)	4 years ago
R.B. Boyer	74d5df7c7a	xds: use envoy's rbac filter to handle intentions entirely within envoy (#8569)	4 years ago
R.B. Boyer	c599a2f5f4	xds: add support for envoy 1.15.0 and drop support for 1.11.x (#8424) ... Related changes: - hard-fail the xDS connection attempt if the envoy version is known to be too old to be supported - remove the RouterMatchSafeRegex proxy feature since all supported envoy versions have it - stop using --max-obj-name-len (due to: envoyproxy/envoy#11740)	4 years ago
Hans Hasselberg	496fb5fc5b	add support for envoy 1.14.4, 1.13.4, 1.12.6 (#8216)	4 years ago
R.B. Boyer	1eef096dfe	xds: version sniff envoy and switch regular expressions from 'regex' to 'safe_regex' on newer envoy versions (#8222) ... - cut down on extra node metadata transmission - split the golden file generation to compare all envoy version	4 years ago
Chris Piraino	735337b170	Append port number to ingress host domain (#8190) ... A port can be sent in the Host header as defined in the HTTP RFC, so we take any hosts that we want to match traffic to and also add another host with the listener port added. Also fix an issue with envoy integration tests not running the case-ingress-gateway-tls test.	4 years ago
Freddy	cc1407e867	Merge http2 integration test case into grpc case (#8164) ... http2 is covered by grpc since grpc uses http2	4 years ago
Hans Hasselberg	e62a43c6cf	Support envoy 1.14.2, 1.13.2, 1.12.4 (#8057)	5 years ago
Chris Piraino	1a853fc954	Always require Host header values for http services (#7990) ... Previously, we did not require the 'service-name.*' host header value when on a single http service was exposed. However, this allows a user to get into a situation where, if they add another service to the listener, suddenly the previous service's traffic might not be routed correctly. Thus, we always require the Host header, even if there is only 1 service. Also, we add the make the default domain matching more restrictive by matching "service-name.ingress.*" by default. This lines up better with the namespace case and more accurately matches the Consul DNS value we expect people to use in this case.	5 years ago
Freddy	9ed325ba8b	Enable gateways to resolve hostnames to IPv4 addresses (#7999) ... The DNS resolution will be handled by Envoy and defaults to LOGICAL_DNS. This discovery type can be overridden on a per-gateway basis with the envoy_dns_discovery_type Gateway Option. If a service contains an instance with a hostname as an address we set the Envoy cluster to use DNS as the discovery type rather than EDS. Since both mesh gateways and terminating gateways route to clusters using SNI, whenever there is a mix of hostnames and IP addresses associated with a service we use the hostname + CDS rather than the IPs + EDS. Note that we detect hostnames by attempting to parse the service instance's address as an IP. If it is not a valid IP we assume it is a hostname.	5 years ago
Daniel Nephin	0a75d32e3e	ci: fix log capture for envoy integration tests ... The previous change, which moved test running to Go, appears to have broken log capturing. I am not entirely sure why, but the run_tests function seems to exit on the first error. This change moves test teardown and log capturing out of run_test, and has the go test runner call them when necessary.	5 years ago
Daniel Nephin	e02ee13657	Make envoy integration tests a `go test` suite (#7842) ... * test/integration: only run against 1 envoy version These tests are slow enough that it seems unlikely that anyone is running multiple versions locally. If someone wants to, a for loop outside of run_test.sh should do the right thing. Remove unused vars. * Remove logic to iterate over test cases, run a single case * Add a golang runner for integration tests * Use build tags for envoy integration tests And add junit-xml report	5 years ago
Kyle Havlovitz	136549205c	Merge pull request #7759 from hashicorp/ingress/tls-hosts ... Add TLS option for Ingress Gateway listeners	5 years ago
Chris Piraino	646902621b	Set default protocol to http in TLS integration test	5 years ago
Daniel Nephin	5655d7f34e	Add outlier_detection check to integration test ... Fix decoding of time.Duration types.	5 years ago
Chris Piraino	5105bf3d67	Require individual services in ingress entry to match protocols (#7774) ... We require any non-wildcard services to match the protocol defined in the listener on write, so that we can maintain a consistent experience through ingress gateways. This also helps guard against accidental misconfiguration by a user. - Update tests that require an updated protocol for ingress gateways	5 years ago
Kyle Havlovitz	b2a0251f66	Add a check for custom host to ingress TLS integration test	5 years ago
Kyle Havlovitz	d452769d92	Add TLS integration test for ingress gateway ... - Pull Consul Root CA from API in order to verify certificate chain - Assert on the DNSSAN as well to ensure it is correct	5 years ago
Kyle Havlovitz	f14c54e25e	Add TLS option and DNS SAN support to ingress config ... xds: Only set TLS context for ingress listener when requested	5 years ago
Chris Piraino	f40833d094	Allow Hosts field to be set on an ingress config entry ... - Validate that this cannot be set on a 'tcp' listener nor on a wildcard service. - Add Hosts field to api and test in consul config write CLI - xds: Configure envoy with user-provided hosts from ingress gateways	5 years ago
Kyle Havlovitz	247f9eaf13	Allow ingress gateways to route traffic based on Host header ... This commit adds the necessary changes to allow an ingress gateway to route traffic from a single defined port to multiple different upstream services in the Consul mesh. To do this, we now require all HTTP requests coming into the ingress gateway to specify a Host header that matches "<service-name>.*" in order to correctly route traffic to the correct service. - Differentiate multiple listener's route names by port - Adds a case in xds for allowing default discovery chains to create a route configuration when on an ingress gateway. This allows default services to easily use host header routing - ingress-gateways have a single route config for each listener that utilizes domain matching to route to different services.	5 years ago
freddygv	eddd5bd73b	PR comments	5 years ago
freddygv	2a85e44519	Add envoy integration tests	5 years ago
Chris Piraino	ecc8a2d6f7	Allow ingress gateways to route through mesh gateways ... - Adds integration test for mesh gateways local + remote modes with ingress - ingress golden files updated for mesh gateway endpoints	5 years ago
Kyle Havlovitz	e7b1ee55de	Add http routing support and integration test to ingress gateways	5 years ago
Kyle Havlovitz	e9e8c0e730	Ingress Gateways for TCP services (#7509) ... * Implements a simple, tcp ingress gateway workflow This adds a new type of gateway for allowing Ingress traffic into Connect from external services. Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>	5 years ago
Pierre Souchay	2199a134a0	More tolerant assert_alive_wan_member_count to fix unstable tests ... Example of failure (very frequent): https://circleci.com/gh/hashicorp/consul/157985	5 years ago
Hans Hasselberg	66415be90e	connect: support envoy 1.14.1 (#7624)	5 years ago
Chris Piraino	584f90bbeb	Fix flapping of mesh gateway connect-service watches (#7575)	5 years ago
Pierre Souchay	eafe9a895a	tests: fixed bats warning (#7544) ... This fixes this bats warning: duplicate test name(s) in /workdir/primary/bats/verify.bats: test_s1_upstream_made_1_connection Test was already defined at line 42, rename it to avoid test name duplication	5 years ago
Hans Hasselberg	6a49a42e98	connect: support for envoy 1.13.1 and 1.12.3 (#7380) ... * setup new envoy versions for CI * bump version on the website too.	5 years ago
R.B. Boyer	6adad71125	wan federation via mesh gateways (#6884) ... This is like a Möbius strip of code due to the fact that low-level components (serf/memberlist) are connected to high-level components (the catalog and mesh-gateways) in a twisty maze of references which make it hard to dive into. With that in mind here's a high level summary of what you'll find in the patch: There are several distinct chunks of code that are affected: * new flags and config options for the server * retry join WAN is slightly different * retry join code is shared to discover primary mesh gateways from secondary datacenters * because retry join logic runs in the *agent* and the results of that operation for primary mesh gateways are needed in the *server* there are some methods like `RefreshPrimaryGatewayFallbackAddresses` that must occur at multiple layers of abstraction just to pass the data down to the right layer. * new cache type `FederationStateListMeshGatewaysName` for use in `proxycfg/xds` layers * the function signature for RPC dialing picked up a new required field (the node name of the destination) * several new RPCs for manipulating a FederationState object: `FederationState:{Apply,Get,List,ListMeshGateways}` * 3 read-only internal APIs for debugging use to invoke those RPCs from curl * raft and fsm changes to persist these FederationStates * replication for FederationStates as they are canonically stored in the Primary and replicated to the Secondaries. * a special derivative of anti-entropy that runs in secondaries to snapshot their local mesh gateway `CheckServiceNodes` and sync them into their upstream FederationState in the primary (this works in conjunction with the replication to distribute addresses for all mesh gateways in all DCs to all other DCs) * a "gateway locator" convenience object to make use of this data to choose the addresses of gateways to use for any given RPC or gossip operation to a remote DC. This gets data from the "retry join" logic in the agent and also directly calls into the FSM. * RPC (`:8300`) on the server sniffs the first byte of a new connection to determine if it's actually doing native TLS. If so it checks the ALPN header for protocol determination (just like how the existing system uses the type-byte marker). * 2 new kinds of protocols are exclusively decoded via this native TLS mechanism: one for ferrying "packet" operations (udp-like) from the gossip layer and one for "stream" operations (tcp-like). The packet operations re-use sockets (using length-prefixing) to cut down on TLS re-negotiation overhead. * the server instances specially wrap the `memberlist.NetTransport` when running with gateway federation enabled (in a `wanfed.Transport`). The general gist is that if it tries to dial a node in the SAME datacenter (deduced by looking at the suffix of the node name) there is no change. If dialing a DIFFERENT datacenter it is wrapped up in a TLS+ALPN blob and sent through some mesh gateways to eventually end up in a server's :8300 port. * a new flag when launching a mesh gateway via `consul connect envoy` to indicate that the servers are to be exposed. This sets a special service meta when registering the gateway into the catalog. * `proxycfg/xds` notice this metadata blob to activate additional watches for the FederationState objects as well as the location of all of the consul servers in that datacenter. * `xds:` if the extra metadata is in place additional clusters are defined in a DC to bulk sink all traffic to another DC's gateways. For the current datacenter we listen on a wildcard name (`server.<dc>.consul`) that load balances all servers as well as one mini-cluster per node (`<node>.server.<dc>.consul`) * the `consul tls cert create` command got a new flag (`-node`) to help create an additional SAN in certs that can be used with this flavor of federation.	5 years ago
Matt Keeler	0041102e29	Change where the envoy snapshots get put when a test fails (#7298) ... This will allow us to capture them in CI	5 years ago
Hans Hasselberg	9cb7adb304	add envoy version 1.12.2 and 1.13.0 to the matrix (#7240) ... * add 1.12.2 * add envoy 1.13.0 * Introduce -envoy-version to get 1.10.0 passing. * update old version and fix consul-exec case * add envoy_version and fix check * Update Envoy CLI tests to account for the 1.13 compatibility changes. Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>	5 years ago
Paschalis Tsilias	a335aa57c5	Expose Envoy's /stats for statsd agents (#7173) ... * Expose Envoy /stats for statsd agents; Add testcases * Remove merge conflict leftover * Add support for prefix instead of path; Fix docstring to mirror these changes * Add new config field to docs; Add testcases to check that /stats/prometheus is exposed as well * Parametrize matchType (prefix or path) and value * Update website/source/docs/connect/proxies/envoy.md Co-Authored-By: Paul Banks <banks@banksco.de> Co-authored-by: Paul Banks <banks@banksco.de>	5 years ago
Matt Keeler	bfc03ec587	Fix a couple bugs regarding intentions with namespaces (#7169)	5 years ago
Matt Keeler	c09693e545	Updates to Config Entries and Connect for Namespaces (#7116)	5 years ago
Chris Piraino	f3b54fa535	Allow configuration of upstream connection limits in Envoy (#6829) ... * Adds 'limits' field to the upstream configuration of a connect proxy This allows a user to configure the envoy connect proxy with 'max_connections', 'max_queued_requests', and 'max_concurrent_requests'. These values are defined in the local proxy on a per-service instance basis and should thus NOT be thought of as a global-level or even service-level value.	5 years ago
R.B. Boyer	2011f3d7dc	xds: mesh gateway CDS requests are now allowed to receive an empty CDS reply (#6787) ... This is the rest of the fix for #6543 that was incompletely fixed in #6576.	5 years ago
Paul Banks	87699eca2f	Fix support for RSA CA keys in Connect. (#6638) ... * Allow RSA CA certs for consul and vault providers to correctly sign EC leaf certs. * Ensure key type ad bits are populated from CA cert and clean up tests * Add integration test and fix error when initializing secondary CA with RSA key. * Add more tests, fix review feedback * Update docs with key type config and output * Apply suggestions from code review Co-Authored-By: R.B. Boyer <rb@hashicorp.com>	5 years ago
R.B. Boyer	8dcba472a2	xds: tcp services using the discovery chain should not assume RDS during LDS (#6623) ... Previously the logic for configuring RDS during LDS for L7 upstreams was overapplied to TCP proxies resulting in a cluster name of <emptystring> being used incorrectly. Fixes #6621	5 years ago
Freddy	fdd10dd8b8	Expose HTTP-based paths through Connect proxy (#6446) ... Fixes: #5396 This PR adds a proxy configuration stanza called expose. These flags register listeners in Connect sidecar proxies to allow requests to specific HTTP paths from outside of the node. This allows services to protect themselves by only listening on the loopback interface, while still accepting traffic from non Connect-enabled services. Under expose there is a boolean checks flag that would automatically expose all registered HTTP and gRPC check paths. This stanza also accepts a paths list to expose individual paths. The primary use case for this functionality would be to expose paths for third parties like Prometheus or the kubelet. Listeners for requests to exposed paths are be configured dynamically at run time. Any time a proxy, or check can be registered, a listener can also be created. In this initial implementation requests to these paths are not authenticated/encrypted.	5 years ago
R.B. Boyer	2cd5a7e542	tests: make envoy integration tests more tolerant of internal retries that may inflate counters (#6539) ... This should remove false positives that look like: cluster.s2.default.primary.*cx_total - expected count: 2, actual count: 3	5 years ago
Pierre Souchay	2f37d68d9b	[BUGFIX][BUILD] When test fail in circle-ci in main, have a proper error message (#6416) ... Since FUNCNAME is not defined when running outside a function, trap does not work and display wrong error message. Example from https://circleci.com/gh/hashicorp/consul/69506 : ``` ⨯ FAIL /home/circleci/project/test/integration/connect/envoy/run-tests.sh: line 1: FUNCNAME[0]: unbound variable make: *** [GNUmakefile:363: test-envoy-integ] Error 1 ``` This fix will avoid this error message and display the real cause.	5 years ago
Matt Keeler	9a5b258edf	Turned on Envoy 1.11.1 integration tests (#6347) ... I also ran this against 1.5.2 so the docs update claiming compatibility should still be accurate.	5 years ago
R.B. Boyer	72207256b9	xds: improve how envoy metrics are emitted (#6312) ... Since generated envoy clusters all are named using (mostly) SNI syntax we can have envoy read the various fields out of that structure and emit it as stats labels to the various telemetry backends. I changed the delimiter for the 'customization hash' from ':' to '~' because ':' is always reencoded by envoy as '_' when generating metrics keys.	5 years ago
R.B. Boyer	8e22d80e35	connect: fix failover through a mesh gateway to a remote datacenter (#6259) ... Failover is pushed entirely down to the data plane by creating envoy clusters and putting each successive destination in a different load assignment priority band. For example this shows that normally requests go to 1.2.3.4:8080 but when that fails they go to 6.7.8.9:8080: - name: foo load_assignment: cluster_name: foo policy: overprovisioning_factor: 100000 endpoints: - priority: 0 lb_endpoints: - endpoint: address: socket_address: address: 1.2.3.4 port_value: 8080 - priority: 1 lb_endpoints: - endpoint: address: socket_address: address: 6.7.8.9 port_value: 8080 Mesh gateways route requests based solely on the SNI header tacked onto the TLS layer. Envoy currently only lets you configure the outbound SNI header at the cluster layer. If you try to failover through a mesh gateway you ideally would configure the SNI value per endpoint, but that's not possible in envoy today. This PR introduces a simpler way around the problem for now: 1. We identify any target of failover that will use mesh gateway mode local or remote and then further isolate any resolver node in the compiled discovery chain that has a failover destination set to one of those targets. 2. For each of these resolvers we will perform a small measurement of comparative healths of the endpoints that come back from the health API for the set of primary target and serial failover targets. We walk the list of targets in order and if any endpoint is healthy we return that target, otherwise we move on to the next target. 3. The CDS and EDS endpoints both perform the measurements in (2) for the affected resolver nodes. 4. For CDS this measurement selects which TLS SNI field to use for the cluster (note the cluster is always going to be named for the primary target) 5. For EDS this measurement selects which set of endpoints will populate the cluster. Priority tiered failover is ignored. One of the big downsides to this approach to failover is that the failover detection and correction is going to be controlled by consul rather than deferring that entirely to the data plane as with the prior version. This also means that we are bound to only failover using official health signals and cannot make use of data plane signals like outlier detection to affect failover. In this specific scenario the lack of data plane signals is ok because the effectiveness is already muted by the fact that the ultimate destination endpoints will have their data plane signals scrambled when they pass through the mesh gateway wrapper anyway so we're not losing much. Another related fix is that we now use the endpoint health from the underlying service, not the health of the gateway (regardless of failover mode).	5 years ago
R.B. Boyer	6393edba53	connect: reconcile how upstream configuration works with discovery chains (#6225) ... * connect: reconcile how upstream configuration works with discovery chains The following upstream config fields for connect sidecars sanely integrate into discovery chain resolution: - Destination Namespace/Datacenter: Compilation occurs locally but using different default values for namespaces and datacenters. The xDS clusters that are created are named as they normally would be. - Mesh Gateway Mode (single upstream): If set this value overrides any value computed for any resolver for the entire discovery chain. The xDS clusters that are created may be named differently (see below). - Mesh Gateway Mode (whole sidecar): If set this value overrides any value computed for any resolver for the entire discovery chain. If this is specifically overridden for a single upstream this value is ignored in that case. The xDS clusters that are created may be named differently (see below). - Protocol (in opaque config): If set this value overrides the value computed when evaluating the entire discovery chain. If the normal chain would be TCP or if this override is set to TCP then the result is that we explicitly disable L7 Routing and Splitting. The xDS clusters that are created may be named differently (see below). - Connect Timeout (in opaque config): If set this value overrides the value for any resolver in the entire discovery chain. The xDS clusters that are created may be named differently (see below). If any of the above overrides affect the actual result of compiling the discovery chain (i.e. "tcp" becomes "grpc" instead of being a no-op override to "tcp") then the relevant parameters are hashed and provided to the xDS layer as a prefix for use in naming the Clusters. This is to ensure that if one Upstream discovery chain has no overrides and tangentially needs a cluster named "api.default.XXX", and another Upstream does have overrides for "api.default.XXX" that they won't cross-pollinate against the operator's wishes. Fixes #6159	5 years ago
Matt Keeler	3053342198	Envoy Mesh Gateway integration tests (#6187) ... * Allow setting the mesh gateway mode for an upstream in config files * Add envoy integration test for mesh gateways This necessitated many supporting changes in most of the other test cases. Add remote mode mesh gateways integration test	5 years ago
R.B. Boyer	e039dfd7f8	connect: rework how the service resolver subset OnlyPassing flag works (#6173) ... The main change is that we no longer filter service instances by health, preferring instead to render all results down into EDS endpoints in envoy and merely label the endpoints as HEALTHY or UNHEALTHY. When OnlyPassing is set to true we will force consul checks in a 'warning' state to render as UNHEALTHY in envoy. Fixes #6171	5 years ago
R.B. Boyer	aca2c5de3f	tests: adding new envoy integration tests for L7 service-resolvers (#6129) ... Additionally: - wait for bootstrap config entries to be applied - run the verify container in the host's PID namespace so we can kill envoys without mounting the docker socket * assert that we actually send HEALTHY and UNHEALTHY endpoints down in EDS during failover	5 years ago
R.B. Boyer	4a9f4b97e6	tests: when running envoy integration tests try to limit container bleedover between cases (#6148)	5 years ago
R.B. Boyer	8a90185bbd	unknown fields now fail, so omit these unimplemented fields (#6125)	5 years ago