217 Commits (b757624b5941517e28e5425e460dfed81239702f)

Author	SHA1	Message	Date
s-christoff	b9a9f395d6	Minor load test fixes (#9394)	4 years ago
s-christoff	348766166e	Allow consul version/consul download url to be inputted via Terraform (#9267)	4 years ago
Freddy	fe728855ed	Add DC and NS support for Envoy metrics (#9207) ... This PR updates the tags that we generate for Envoy stats. Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.	4 years ago
Mike Morris	7af643ac37	ci: update to Go 1.15.4 and alpine:3.12 (#9036) ... * ci: stop building darwin/386 binaries Go 1.15 drops support for 32-bit binaries on Darwin https://golang.org/doc/go1.15#darwin * tls: ConnectionState::NegotiatedProtocolIsMutual is deprecated in Go 1.15, this value is always true * correct error messages that changed slightly * Completely regenerate some TLS test data Co-authored-by: R.B. Boyer <rb@hashicorp.com>	4 years ago
R.B. Boyer	5afd04897c	test: use direct service registration in envoy integration tests (#9138) ... This has the biggest impact on enterprise test cases that use namespaced registrations, which prior to this change sometimes failed the initial registration because the namespace was not yet created.	4 years ago
R.B. Boyer	8baf158ea8	Revert "Add namespace support for metrics (OSS) (#9117)" (#9124) ... This reverts commit 06b3b017d3.	4 years ago
Freddy	06b3b017d3	Add namespace support for metrics (OSS) (#9117)	4 years ago
Aaron Lane	ef90db1943	Merge pull request #9112 from hashicorp/aaron-lane-patch-2 ... Update loadtest AMI name, description	4 years ago
Aaron Lane	78de48ef42	Update loadtest AMI name, description ... This commit updates the Packer properties `ami_name` and `ami_description` for the loadtest image to reflect the image intent.	4 years ago
Aaron Lane	665019e884	Link to packer directory from terraform ReadMe	4 years ago
R.B. Boyer	44bd8bd2ae	use the docker proxy for more envoy integration test containers (#9085)	4 years ago
R.B. Boyer	957293d884	wait_for_namespace should take two args (#9086)	4 years ago
Alvin Huang	ae6185a554	use hashicorp docker mirror in envoy helper (#9080)	4 years ago
R.B. Boyer	d7b37e3c6e	fix envoy integ test wait_for_namespace to actually work on CI (#9082)	4 years ago
Alvin Huang	ea88956c9c	use hashicorp docker mirror to prevent rate limit (#9070)	4 years ago
R.B. Boyer	a66c4591d7	agent: introduce path allow list for requests going through the metrics proxy (#9059) ... Added a new option `ui_config.metrics_proxy.path_allowlist`. This defaults to `["/api/v1/query", "/api/v1/query_range"]` when the metrics provider is set to `prometheus`. Requests that do not use one of the allow-listed paths (via exact match) get a 403 Forbidden response instead.	4 years ago
R.B. Boyer	b724e096c2	add namespace waiting function to envoy integration tests (#9051)	4 years ago
R.B. Boyer	8e70c3fc25	missed adding the test delay to the l7-intentions envoy integration test (#9052)	4 years ago
R.B. Boyer	d7c7858d87	Fix even more test flakes in intentions related envoy integration tests (#9013) ... The key thing here is to use `curl --no-keepalive` so that envoy pre-1.15 tests will reliably use the latest listener every time. Extra: - Switched away from editing line-item intentions the legacy way. - Removed some teardown scripts, as we don't share anything between cases anyway - Removed unnecessary use of `run` in some places.	4 years ago
R.B. Boyer	934c65ad77	fix flaky envoy integration tests involving intentions (#8996) ... There is a delay between an intentions change being made, and it being reflected in the Envoy runtime configuration. Now that the enforcement happens inside of Envoy instead of over in the agent, our tests need to explicitly wait until the xDS reconfiguration is complete before attempting to assert intentions worked. Also remove a few double retry loops.	4 years ago
R.B. Boyer	a2c50d3303	connect: add support for envoy 1.16.0, drop support for 1.12.x, and bump point releases as well (#8944) ... Supported versions will be: "1.16.0", "1.15.2", "1.14.5", "1.13.6"	4 years ago
R.B. Boyer	0bf62246e5	speed up envoy integration tests by removing docker-compose (#8982) ... This speeds up individual envoy integration test runs from ~23m to ~14m. It's also a pre-req for possibly switching to doing the tests entirely within Go (no shell-outs).	4 years ago
R.B. Boyer	da4ec9ff27	restore the discovery of tests cases by file system existence (#8983)	4 years ago
R.B. Boyer	a68c0ce1e7	speed up envoy integ tests by not politely stopping containers before destroying them (#8969) ... In local testing this sped up the stop_services call from 11s to 1s per test.	4 years ago
R.B. Boyer	1b413b0444	connect: support defining intentions using layer 7 criteria (#8839) ... Extend Consul’s intentions model to allow for request-based access control enforcement for HTTP-like protocols in addition to the existing connection-based enforcement for unspecified protocols (e.g. tcp).	4 years ago
s-christoff	5da640b287	Add load testing framework (#8571)	4 years ago
R.B. Boyer	9801ef8eb1	agent: enable enable_central_service_config by default (#8746)	4 years ago
Jack	9e1c6727f9	Add http2 and grpc support to ingress gateways (#8458)	4 years ago
R.B. Boyer	74d5df7c7a	xds: use envoy's rbac filter to handle intentions entirely within envoy (#8569)	4 years ago
R.B. Boyer	c599a2f5f4	xds: add support for envoy 1.15.0 and drop support for 1.11.x (#8424) ... Related changes: - hard-fail the xDS connection attempt if the envoy version is known to be too old to be supported - remove the RouterMatchSafeRegex proxy feature since all supported envoy versions have it - stop using --max-obj-name-len (due to: envoyproxy/envoy#11740)	4 years ago
Hans Hasselberg	496fb5fc5b	add support for envoy 1.14.4, 1.13.4, 1.12.6 (#8216)	4 years ago
R.B. Boyer	1eef096dfe	xds: version sniff envoy and switch regular expressions from 'regex' to 'safe_regex' on newer envoy versions (#8222) ... - cut down on extra node metadata transmission - split the golden file generation to compare all envoy version	4 years ago
Chris Piraino	735337b170	Append port number to ingress host domain (#8190) ... A port can be sent in the Host header as defined in the HTTP RFC, so we take any hosts that we want to match traffic to and also add another host with the listener port added. Also fix an issue with envoy integration tests not running the case-ingress-gateway-tls test.	4 years ago
Freddy	cc1407e867	Merge http2 integration test case into grpc case (#8164) ... http2 is covered by grpc since grpc uses http2	4 years ago
Hans Hasselberg	e62a43c6cf	Support envoy 1.14.2, 1.13.2, 1.12.4 (#8057)	5 years ago
Chris Piraino	1a853fc954	Always require Host header values for http services (#7990) ... Previously, we did not require the 'service-name.*' host header value when on a single http service was exposed. However, this allows a user to get into a situation where, if they add another service to the listener, suddenly the previous service's traffic might not be routed correctly. Thus, we always require the Host header, even if there is only 1 service. Also, we add the make the default domain matching more restrictive by matching "service-name.ingress.*" by default. This lines up better with the namespace case and more accurately matches the Consul DNS value we expect people to use in this case.	5 years ago
Freddy	9ed325ba8b	Enable gateways to resolve hostnames to IPv4 addresses (#7999) ... The DNS resolution will be handled by Envoy and defaults to LOGICAL_DNS. This discovery type can be overridden on a per-gateway basis with the envoy_dns_discovery_type Gateway Option. If a service contains an instance with a hostname as an address we set the Envoy cluster to use DNS as the discovery type rather than EDS. Since both mesh gateways and terminating gateways route to clusters using SNI, whenever there is a mix of hostnames and IP addresses associated with a service we use the hostname + CDS rather than the IPs + EDS. Note that we detect hostnames by attempting to parse the service instance's address as an IP. If it is not a valid IP we assume it is a hostname.	5 years ago
Daniel Nephin	0a75d32e3e	ci: fix log capture for envoy integration tests ... The previous change, which moved test running to Go, appears to have broken log capturing. I am not entirely sure why, but the run_tests function seems to exit on the first error. This change moves test teardown and log capturing out of run_test, and has the go test runner call them when necessary.	5 years ago
Daniel Nephin	e02ee13657	Make envoy integration tests a `go test` suite (#7842) ... * test/integration: only run against 1 envoy version These tests are slow enough that it seems unlikely that anyone is running multiple versions locally. If someone wants to, a for loop outside of run_test.sh should do the right thing. Remove unused vars. * Remove logic to iterate over test cases, run a single case * Add a golang runner for integration tests * Use build tags for envoy integration tests And add junit-xml report	5 years ago
Kyle Havlovitz	136549205c	Merge pull request #7759 from hashicorp/ingress/tls-hosts ... Add TLS option for Ingress Gateway listeners	5 years ago
Chris Piraino	646902621b	Set default protocol to http in TLS integration test	5 years ago
Daniel Nephin	5655d7f34e	Add outlier_detection check to integration test ... Fix decoding of time.Duration types.	5 years ago
Chris Piraino	5105bf3d67	Require individual services in ingress entry to match protocols (#7774) ... We require any non-wildcard services to match the protocol defined in the listener on write, so that we can maintain a consistent experience through ingress gateways. This also helps guard against accidental misconfiguration by a user. - Update tests that require an updated protocol for ingress gateways	5 years ago
Kyle Havlovitz	b2a0251f66	Add a check for custom host to ingress TLS integration test	5 years ago
Kyle Havlovitz	d452769d92	Add TLS integration test for ingress gateway ... - Pull Consul Root CA from API in order to verify certificate chain - Assert on the DNSSAN as well to ensure it is correct	5 years ago
Kyle Havlovitz	f14c54e25e	Add TLS option and DNS SAN support to ingress config ... xds: Only set TLS context for ingress listener when requested	5 years ago
Chris Piraino	f40833d094	Allow Hosts field to be set on an ingress config entry ... - Validate that this cannot be set on a 'tcp' listener nor on a wildcard service. - Add Hosts field to api and test in consul config write CLI - xds: Configure envoy with user-provided hosts from ingress gateways	5 years ago
Kyle Havlovitz	247f9eaf13	Allow ingress gateways to route traffic based on Host header ... This commit adds the necessary changes to allow an ingress gateway to route traffic from a single defined port to multiple different upstream services in the Consul mesh. To do this, we now require all HTTP requests coming into the ingress gateway to specify a Host header that matches "<service-name>.*" in order to correctly route traffic to the correct service. - Differentiate multiple listener's route names by port - Adds a case in xds for allowing default discovery chains to create a route configuration when on an ingress gateway. This allows default services to easily use host header routing - ingress-gateways have a single route config for each listener that utilizes domain matching to route to different services.	5 years ago
freddygv	eddd5bd73b	PR comments	5 years ago
freddygv	2a85e44519	Add envoy integration tests	5 years ago
Chris Piraino	ecc8a2d6f7	Allow ingress gateways to route through mesh gateways ... - Adds integration test for mesh gateways local + remote modes with ingress - ingress golden files updated for mesh gateway endpoints	5 years ago
Kyle Havlovitz	e7b1ee55de	Add http routing support and integration test to ingress gateways	5 years ago
Kyle Havlovitz	e9e8c0e730	Ingress Gateways for TCP services (#7509) ... * Implements a simple, tcp ingress gateway workflow This adds a new type of gateway for allowing Ingress traffic into Connect from external services. Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>	5 years ago
Pierre Souchay	2199a134a0	More tolerant assert_alive_wan_member_count to fix unstable tests ... Example of failure (very frequent): https://circleci.com/gh/hashicorp/consul/157985	5 years ago
Hans Hasselberg	66415be90e	connect: support envoy 1.14.1 (#7624)	5 years ago
Chris Piraino	584f90bbeb	Fix flapping of mesh gateway connect-service watches (#7575)	5 years ago
Pierre Souchay	eafe9a895a	tests: fixed bats warning (#7544) ... This fixes this bats warning: duplicate test name(s) in /workdir/primary/bats/verify.bats: test_s1_upstream_made_1_connection Test was already defined at line 42, rename it to avoid test name duplication	5 years ago
Hans Hasselberg	6a49a42e98	connect: support for envoy 1.13.1 and 1.12.3 (#7380) ... * setup new envoy versions for CI * bump version on the website too.	5 years ago
R.B. Boyer	6adad71125	wan federation via mesh gateways (#6884) ... This is like a Möbius strip of code due to the fact that low-level components (serf/memberlist) are connected to high-level components (the catalog and mesh-gateways) in a twisty maze of references which make it hard to dive into. With that in mind here's a high level summary of what you'll find in the patch: There are several distinct chunks of code that are affected: * new flags and config options for the server * retry join WAN is slightly different * retry join code is shared to discover primary mesh gateways from secondary datacenters * because retry join logic runs in the *agent* and the results of that operation for primary mesh gateways are needed in the *server* there are some methods like `RefreshPrimaryGatewayFallbackAddresses` that must occur at multiple layers of abstraction just to pass the data down to the right layer. * new cache type `FederationStateListMeshGatewaysName` for use in `proxycfg/xds` layers * the function signature for RPC dialing picked up a new required field (the node name of the destination) * several new RPCs for manipulating a FederationState object: `FederationState:{Apply,Get,List,ListMeshGateways}` * 3 read-only internal APIs for debugging use to invoke those RPCs from curl * raft and fsm changes to persist these FederationStates * replication for FederationStates as they are canonically stored in the Primary and replicated to the Secondaries. * a special derivative of anti-entropy that runs in secondaries to snapshot their local mesh gateway `CheckServiceNodes` and sync them into their upstream FederationState in the primary (this works in conjunction with the replication to distribute addresses for all mesh gateways in all DCs to all other DCs) * a "gateway locator" convenience object to make use of this data to choose the addresses of gateways to use for any given RPC or gossip operation to a remote DC. This gets data from the "retry join" logic in the agent and also directly calls into the FSM. * RPC (`:8300`) on the server sniffs the first byte of a new connection to determine if it's actually doing native TLS. If so it checks the ALPN header for protocol determination (just like how the existing system uses the type-byte marker). * 2 new kinds of protocols are exclusively decoded via this native TLS mechanism: one for ferrying "packet" operations (udp-like) from the gossip layer and one for "stream" operations (tcp-like). The packet operations re-use sockets (using length-prefixing) to cut down on TLS re-negotiation overhead. * the server instances specially wrap the `memberlist.NetTransport` when running with gateway federation enabled (in a `wanfed.Transport`). The general gist is that if it tries to dial a node in the SAME datacenter (deduced by looking at the suffix of the node name) there is no change. If dialing a DIFFERENT datacenter it is wrapped up in a TLS+ALPN blob and sent through some mesh gateways to eventually end up in a server's :8300 port. * a new flag when launching a mesh gateway via `consul connect envoy` to indicate that the servers are to be exposed. This sets a special service meta when registering the gateway into the catalog. * `proxycfg/xds` notice this metadata blob to activate additional watches for the FederationState objects as well as the location of all of the consul servers in that datacenter. * `xds:` if the extra metadata is in place additional clusters are defined in a DC to bulk sink all traffic to another DC's gateways. For the current datacenter we listen on a wildcard name (`server.<dc>.consul`) that load balances all servers as well as one mini-cluster per node (`<node>.server.<dc>.consul`) * the `consul tls cert create` command got a new flag (`-node`) to help create an additional SAN in certs that can be used with this flavor of federation.	5 years ago
Matt Keeler	0041102e29	Change where the envoy snapshots get put when a test fails (#7298) ... This will allow us to capture them in CI	5 years ago
Hans Hasselberg	9cb7adb304	add envoy version 1.12.2 and 1.13.0 to the matrix (#7240) ... * add 1.12.2 * add envoy 1.13.0 * Introduce -envoy-version to get 1.10.0 passing. * update old version and fix consul-exec case * add envoy_version and fix check * Update Envoy CLI tests to account for the 1.13 compatibility changes. Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>	5 years ago
Paschalis Tsilias	a335aa57c5	Expose Envoy's /stats for statsd agents (#7173) ... * Expose Envoy /stats for statsd agents; Add testcases * Remove merge conflict leftover * Add support for prefix instead of path; Fix docstring to mirror these changes * Add new config field to docs; Add testcases to check that /stats/prometheus is exposed as well * Parametrize matchType (prefix or path) and value * Update website/source/docs/connect/proxies/envoy.md Co-Authored-By: Paul Banks <banks@banksco.de> Co-authored-by: Paul Banks <banks@banksco.de>	5 years ago
Matt Keeler	bfc03ec587	Fix a couple bugs regarding intentions with namespaces (#7169)	5 years ago
Matt Keeler	c09693e545	Updates to Config Entries and Connect for Namespaces (#7116)	5 years ago
Chris Piraino	f3b54fa535	Allow configuration of upstream connection limits in Envoy (#6829) ... * Adds 'limits' field to the upstream configuration of a connect proxy This allows a user to configure the envoy connect proxy with 'max_connections', 'max_queued_requests', and 'max_concurrent_requests'. These values are defined in the local proxy on a per-service instance basis and should thus NOT be thought of as a global-level or even service-level value.	5 years ago
R.B. Boyer	2011f3d7dc	xds: mesh gateway CDS requests are now allowed to receive an empty CDS reply (#6787) ... This is the rest of the fix for #6543 that was incompletely fixed in #6576.	5 years ago
Paul Banks	87699eca2f	Fix support for RSA CA keys in Connect. (#6638) ... * Allow RSA CA certs for consul and vault providers to correctly sign EC leaf certs. * Ensure key type ad bits are populated from CA cert and clean up tests * Add integration test and fix error when initializing secondary CA with RSA key. * Add more tests, fix review feedback * Update docs with key type config and output * Apply suggestions from code review Co-Authored-By: R.B. Boyer <rb@hashicorp.com>	5 years ago
R.B. Boyer	8dcba472a2	xds: tcp services using the discovery chain should not assume RDS during LDS (#6623) ... Previously the logic for configuring RDS during LDS for L7 upstreams was overapplied to TCP proxies resulting in a cluster name of <emptystring> being used incorrectly. Fixes #6621	5 years ago
Freddy	fdd10dd8b8	Expose HTTP-based paths through Connect proxy (#6446) ... Fixes: #5396 This PR adds a proxy configuration stanza called expose. These flags register listeners in Connect sidecar proxies to allow requests to specific HTTP paths from outside of the node. This allows services to protect themselves by only listening on the loopback interface, while still accepting traffic from non Connect-enabled services. Under expose there is a boolean checks flag that would automatically expose all registered HTTP and gRPC check paths. This stanza also accepts a paths list to expose individual paths. The primary use case for this functionality would be to expose paths for third parties like Prometheus or the kubelet. Listeners for requests to exposed paths are be configured dynamically at run time. Any time a proxy, or check can be registered, a listener can also be created. In this initial implementation requests to these paths are not authenticated/encrypted.	5 years ago
R.B. Boyer	2cd5a7e542	tests: make envoy integration tests more tolerant of internal retries that may inflate counters (#6539) ... This should remove false positives that look like: cluster.s2.default.primary.*cx_total - expected count: 2, actual count: 3	5 years ago
Pierre Souchay	2f37d68d9b	[BUGFIX][BUILD] When test fail in circle-ci in main, have a proper error message (#6416) ... Since FUNCNAME is not defined when running outside a function, trap does not work and display wrong error message. Example from https://circleci.com/gh/hashicorp/consul/69506 : ``` ⨯ FAIL /home/circleci/project/test/integration/connect/envoy/run-tests.sh: line 1: FUNCNAME[0]: unbound variable make: *** [GNUmakefile:363: test-envoy-integ] Error 1 ``` This fix will avoid this error message and display the real cause.	5 years ago
Matt Keeler	9a5b258edf	Turned on Envoy 1.11.1 integration tests (#6347) ... I also ran this against 1.5.2 so the docs update claiming compatibility should still be accurate.	5 years ago
R.B. Boyer	72207256b9	xds: improve how envoy metrics are emitted (#6312) ... Since generated envoy clusters all are named using (mostly) SNI syntax we can have envoy read the various fields out of that structure and emit it as stats labels to the various telemetry backends. I changed the delimiter for the 'customization hash' from ':' to '~' because ':' is always reencoded by envoy as '_' when generating metrics keys.	5 years ago
R.B. Boyer	8e22d80e35	connect: fix failover through a mesh gateway to a remote datacenter (#6259) ... Failover is pushed entirely down to the data plane by creating envoy clusters and putting each successive destination in a different load assignment priority band. For example this shows that normally requests go to 1.2.3.4:8080 but when that fails they go to 6.7.8.9:8080: - name: foo load_assignment: cluster_name: foo policy: overprovisioning_factor: 100000 endpoints: - priority: 0 lb_endpoints: - endpoint: address: socket_address: address: 1.2.3.4 port_value: 8080 - priority: 1 lb_endpoints: - endpoint: address: socket_address: address: 6.7.8.9 port_value: 8080 Mesh gateways route requests based solely on the SNI header tacked onto the TLS layer. Envoy currently only lets you configure the outbound SNI header at the cluster layer. If you try to failover through a mesh gateway you ideally would configure the SNI value per endpoint, but that's not possible in envoy today. This PR introduces a simpler way around the problem for now: 1. We identify any target of failover that will use mesh gateway mode local or remote and then further isolate any resolver node in the compiled discovery chain that has a failover destination set to one of those targets. 2. For each of these resolvers we will perform a small measurement of comparative healths of the endpoints that come back from the health API for the set of primary target and serial failover targets. We walk the list of targets in order and if any endpoint is healthy we return that target, otherwise we move on to the next target. 3. The CDS and EDS endpoints both perform the measurements in (2) for the affected resolver nodes. 4. For CDS this measurement selects which TLS SNI field to use for the cluster (note the cluster is always going to be named for the primary target) 5. For EDS this measurement selects which set of endpoints will populate the cluster. Priority tiered failover is ignored. One of the big downsides to this approach to failover is that the failover detection and correction is going to be controlled by consul rather than deferring that entirely to the data plane as with the prior version. This also means that we are bound to only failover using official health signals and cannot make use of data plane signals like outlier detection to affect failover. In this specific scenario the lack of data plane signals is ok because the effectiveness is already muted by the fact that the ultimate destination endpoints will have their data plane signals scrambled when they pass through the mesh gateway wrapper anyway so we're not losing much. Another related fix is that we now use the endpoint health from the underlying service, not the health of the gateway (regardless of failover mode).	5 years ago
R.B. Boyer	6393edba53	connect: reconcile how upstream configuration works with discovery chains (#6225) ... * connect: reconcile how upstream configuration works with discovery chains The following upstream config fields for connect sidecars sanely integrate into discovery chain resolution: - Destination Namespace/Datacenter: Compilation occurs locally but using different default values for namespaces and datacenters. The xDS clusters that are created are named as they normally would be. - Mesh Gateway Mode (single upstream): If set this value overrides any value computed for any resolver for the entire discovery chain. The xDS clusters that are created may be named differently (see below). - Mesh Gateway Mode (whole sidecar): If set this value overrides any value computed for any resolver for the entire discovery chain. If this is specifically overridden for a single upstream this value is ignored in that case. The xDS clusters that are created may be named differently (see below). - Protocol (in opaque config): If set this value overrides the value computed when evaluating the entire discovery chain. If the normal chain would be TCP or if this override is set to TCP then the result is that we explicitly disable L7 Routing and Splitting. The xDS clusters that are created may be named differently (see below). - Connect Timeout (in opaque config): If set this value overrides the value for any resolver in the entire discovery chain. The xDS clusters that are created may be named differently (see below). If any of the above overrides affect the actual result of compiling the discovery chain (i.e. "tcp" becomes "grpc" instead of being a no-op override to "tcp") then the relevant parameters are hashed and provided to the xDS layer as a prefix for use in naming the Clusters. This is to ensure that if one Upstream discovery chain has no overrides and tangentially needs a cluster named "api.default.XXX", and another Upstream does have overrides for "api.default.XXX" that they won't cross-pollinate against the operator's wishes. Fixes #6159	5 years ago
Matt Keeler	3053342198	Envoy Mesh Gateway integration tests (#6187) ... * Allow setting the mesh gateway mode for an upstream in config files * Add envoy integration test for mesh gateways This necessitated many supporting changes in most of the other test cases. Add remote mode mesh gateways integration test	5 years ago
R.B. Boyer	e039dfd7f8	connect: rework how the service resolver subset OnlyPassing flag works (#6173) ... The main change is that we no longer filter service instances by health, preferring instead to render all results down into EDS endpoints in envoy and merely label the endpoints as HEALTHY or UNHEALTHY. When OnlyPassing is set to true we will force consul checks in a 'warning' state to render as UNHEALTHY in envoy. Fixes #6171	5 years ago
R.B. Boyer	aca2c5de3f	tests: adding new envoy integration tests for L7 service-resolvers (#6129) ... Additionally: - wait for bootstrap config entries to be applied - run the verify container in the host's PID namespace so we can kill envoys without mounting the docker socket * assert that we actually send HEALTHY and UNHEALTHY endpoints down in EDS during failover	5 years ago
R.B. Boyer	4a9f4b97e6	tests: when running envoy integration tests try to limit container bleedover between cases (#6148)	5 years ago
R.B. Boyer	8a90185bbd	unknown fields now fail, so omit these unimplemented fields (#6125)	5 years ago
R.B. Boyer	9138a97054	Fix bug in service-resolver redirects if the destination uses a default resolver. (#6122) ... Also: - add back an internal http endpoint to dump a compiled discovery chain for debugging purposes Before the CompiledDiscoveryChain.IsDefault() method would test: - is this chain just one resolver step? - is that resolver step just the default? But what I forgot to test: - is that resolver step for the same service that the chain represents? This last point is important because if you configured just one config entry: kind = "service-resolver" name = "web" redirect { service = "other" } and requested the chain for "web" you'd get back a **default** resolver for "other". In the xDS code the IsDefault() method is used to determine if this chain is "empty". If it is then we use the pre-discovery-chain logic that just uses data embedded in the Upstream object (and still lets the escape hatches function). In the example above that means certain parts of the xDS code were going to try referencing a cluster named "web..." despite the other parts of the xDS code maintaining clusters named "other...".	5 years ago
R.B. Boyer	911ed76e5b	tests: further reduce envoy integration test flakiness (#6112) ... In addition to waiting until s2 shows up healthy in the Catalog, wait until s2 endpoints show up healthy via EDS in the s1 upstream clusters.	5 years ago
R.B. Boyer	d4e58e9773	test: for envoy integration tests bump the time to wait for the upstream to be healthy (#6109)	5 years ago
R.B. Boyer	20caa4f744	test: for envoy integration tests, wait until 's2' is healthy in consul before interrogating envoy (#6108) ... When the envoy healthy panic threshold was explicitly disabled as part of L7 traffic management it changed how envoy decided to load balance to endpoints in a cluster. This only matters when envoy is in "panic mode" aka "when you have a bunch of unhealthy endpoints". Panic mode sends traffic to unhealthy instances in certain circumstances. Note: Prior to explicitly disabling the healthy panic threshold, the default value is 50%. What was happening is that the test harness was bringing up consul the sidecars, and the service instances all at once and sometimes the proxies wouldn't have time to be checked by consul to be labeled as 'passing' in the catalog before a round of EDS happened. The xDS server in consul effectively queries /v1/health/connect/s2 and gets 1 result, but that one result has a 'critical' check so the xDS server sends back that endpoint labeled as UNHEALTHY. Envoy sees that 100% of the endpoints in the cluster are unhealthy and would enter panic mode and still send traffic to s2. This is why the test suites PRIOR to disabling the healthy panic threshold worked. They were _incorrectly_ passing. When the healthy panic threshol is disabled, envoy never enters panic mode in this situation and thus the cluster has zero healthy endpoints so load balancing goes nowhere and the tests fail. Why does this only affect the test suites for envoy 1.8.0? My guess is that https://github.com/envoyproxy/envoy/pull/4442 was merged into the 1.9.x series and somehow that plays a role. This PR modifies the bats scripts to explicitly wait until the upstream sidecar is healthy as measured by /v1/health/connect/s2?passing BEFORE trying to interrogate envoy which should make the tests less racy.	5 years ago
Jack Pearkes	e6f1b78efb	Make cluster names SNI always (#6081) ... * Make cluster names SNI always * Update some tests * Ensure we check for prepared query types * Use sni for route cluster names * Proper mesh gateway mode defaulting when the discovery chain is used * Ignore service splits from PatchSliceOfMaps * Update some xds golden files for proper test output * Allow for grpc/http listeners/cluster configs with the disco chain * Update stats expectation	5 years ago
R.B. Boyer	4bdb690a25	activate most discovery chain features in xDS for envoy (#6024)	5 years ago
Hans Hasselberg	33a7df3330	tls: auto_encrypt enables automatic RPC cert provisioning for consul clients (#5597)	5 years ago
Paul Banks	9f656a2dc8	Fix envoy 1.10 exec (#5964) ... * Make exec test assert Envoy version - it was not rebuilding before and so often ran against wrong version. This makes 1.10 fail consistenty. * Switch Envoy exec to use a named pipe rather than FD magic since Envoy 1.10 doesn't support that. * Refactor to use an internal shim command for piping the bootstrap through. * Fmt. So sad that vscode golang fails so often these days. * go mod tidy * revert go mod tidy changes * Revert "ignore consul-exec tests until fixed (#5986)" This reverts commit 683262a686. * Review cleanups	6 years ago
Alvin Huang	683262a686	ignore consul-exec tests until fixed (#5986)	6 years ago
Paul Banks	ffcfdf29fc	Upgrade xDS (go-control-plane) API to support Envoy 1.10. (#5872) ... * Upgrade xDS (go-control-plane) API to support Envoy 1.10. This includes backwards compatibility shim to work around the ext_authz package rename in 1.10. It also adds integration test support in CI for 1.10.0. * Fix go vet complaints * go mod vendor * Update Envoy version info in docs * Update website/source/docs/connect/proxies/envoy.md	6 years ago
Paul Banks	2d47b28722	Envoy integration test improvements (#5797) ... * Grab consul logs on integration test failures too and don't remove .gitignore * Don't wipe logs so we have some artifacts to upload at the end	6 years ago
Alvin Huang	19f44cd1cc	remove container after docker run exits (#5798)	6 years ago
Paul Banks	446abd7aa2	Make central conf test work when run in a suite. (#5767) ... * Make central conf test work when run in a suite. This switches integration tests to hard restart Consul each time which causes less surpise when some tests need to set configs that don't work on consul reload. This also increases the isolation and repeatability of the tests by dropping Consul's state entirely for each case run. * Remove aborted attempt to make restart optional.	6 years ago
Paul Banks	0cfb6051ea	Add integration test for central config; fix central config WIP (#5752) ... * Add integration test for central config; fix central config WIP * Add integration test for central config; fix central config WIP * Set proxy protocol correctly and begin adding upstream support * Add upstreams to service config cache key and start new notify watcher if they change. This doesn't update the tests to pass though. * Fix some merging logic get things working manually with a hack (TODO fix properly) * Simplification to not allow enabling sidecars centrally - it makes no sense without upstreams anyway * Test compile again and obvious ones pass. Lots of failures locally not debugged yet but may be flakes. Pushing up to see what CI does * Fix up service manageer and API test failures * Remove the enable command since it no longer makes much sense without being able to turn on sidecar proxies centrally * Remove version.go hack - will make integration test fail until release * Remove unused code from commands and upstream merge * Re-bump version to 1.5.0	6 years ago
Paul Banks	421ecd32fc	Connect: allow configuring Envoy for L7 Observability (#5558) ... * Add support for HTTP proxy listeners * Add customizable bootstrap configuration options * Debug logging for xDS AuthZ * Add Envoy Integration test suite with basic test coverage * Add envoy command tests to cover new cases * Add tracing integration test * Add gRPC support WIP * Merged changes from master Docker. get CI integration to work with same Dockerfile now * Make docker build optional for integration * Enable integration tests again! * http2 and grpc integration tests and fixes * Fix up command config tests * Store all container logs as artifacts in circle on fail * Add retries to outer part of stats measurements as we keep missing them in CI * Only dump logs on failing cases * Fix typos from code review * Review tidying and make tests pass again * Add debug logs to exec test. * Fix legit test failure caused by upstream rename in envoy config * Attempt to reduce cases of bad TLS handshake in CI integration tests * bring up the right service * Add prometheus integration test * Add test for denied AuthZ both HTTP and TCP * Try ANSI term for Circle	6 years ago
Hans Hasselberg	0fc1c203cc	snapshot: read meta.json correctly. (#5193) ... * snapshot: read meta.json correctly. Fixes #4452.	6 years ago
Paul Banks	bbde408b09	Update test certificates that expire this year to be way in the future	7 years ago
James Phillips	93f68555d0	Adds enable_agent_tls_for_checks configuration option which allows (#3661) ... HTTP health checks for services requiring 2-way TLS to be checked using the agent's credentials.	7 years ago
Frank Schroeder	c94751ad43	test: replace porter tool with freeport lib ... This patch removes the porter tool which hands out free ports from a given range with a library which does the same thing. The challenge for acquiring free ports in concurrent go test runs is that go packages are tested concurrently and run in separate processes. There has to be some inter-process synchronization in preventing processes allocating the same ports. freeport allocates blocks of ports from a range expected to be not in heavy use and implements a system-wide mutex by binding to the first port of that block for the lifetime of the application. Ports are then provided sequentially from that block and are tested on localhost before being returned as available.	7 years ago
Alex Dadgar	c7a65b4587	Testutil falls back to random ports w/o porter (#3604) ... * Testutil falls back to random ports w/o porter This PR allows the testutil server to be used without porter. * Adds sterner-sounding fallback comments.	7 years ago
Frank Schroeder	86901b887d	porter: add better warning if missing	7 years ago
James Phillips	d0e099e54c	Makes porter take over if an existing instance died.	7 years ago
James Phillips	e19f547529	Makes porter more conservative by trying to connect to ports before handing them out.	7 years ago
Frank Schröder	12216583a1	New config parser, HCL support, multiple bind addrs (#3480) ... * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt	7 years ago
Frank Schroeder	01bc7dd3c4	test: log exit code in cluster.bash	8 years ago
Frank Schroeder	bab941e76f	test: add script for starting a multi-node cluster	8 years ago
James Phillips	6f7ff554d0	Updates unit test certs for another year.	8 years ago
James Phillips	dd85930b6d	Updates expired test certs and includes a script to generate new certs.	8 years ago
Kyle Havlovitz	ae6bf56ee1	Add tls client options to api/cli	8 years ago
Kyle Havlovitz	197dc10a7f	Add utility types to enable checking for unset flags	8 years ago
James Phillips	c01a3871c9	Adds support for snapshots and restores. (#2396) ... * Updates Raft library to get new snapshot/restore API. * Basic backup and restore working, but need some cleanup. * Breaks out a snapshot module and adds a SHA256 integrity check. * Adds snapshot ACL and fills in some missing comments. * Require a consistent read for snapshots. * Make sure snapshot works if ACLs aren't enabled. * Adds a bit of package documentation. * Returns an empty response from restore to avoid EOF errors. * Adds API client support for snapshots. * Makes internal file names match on-disk file snapshots. * Adds DC and token coverage for snapshot API test. * Adds missing documentation. * Adds a unit test for the snapshot client endpoint. * Moves the connection pool out of the client for easier testing. * Fixes an incidental issue in the prepared query unit test. I realized I had two servers in bootstrap mode so this wasn't a good setup. * Adds a half close to the TCP stream and fixes panic on error. * Adds client and endpoint tests for snapshots. * Moves the pool back into the snapshot RPC client. * Adds a TLS test and fixes half-closes for TLS connections. * Tweaks some comments. * Adds a low-level snapshot test. This is independent of Consul so we can pull this out into a library later if we want to. * Cleans up snapshot and archive and completes archive tests. * Sends a clear error for snapshot operations in dev mode. Snapshots require the Raft snapshots to be readable, which isn't supported in dev mode. Send a clear error instead of a deep-down Raft one. * Adds docs for the snapshot endpoint. * Adds a stale mode and index feedback for snapshot saves. This gives folks a way to extract data even if the cluster has no leader. * Changes the internal format of a snapshot from zip to tgz. * Pulls in Raft fix to cancel inflight before a restore. * Pulls in new Raft restore interface. * Adds metadata to snapshot saves and a verify function. * Adds basic save and restore snapshot CLI commands. * Gets rid of tarball extensions and adds restore message. * Fixes an incidental bad link in the KV docs. * Adds documentation for the snapshot CLI commands. * Scuttle any request body when a snapshot is saved. * Fixes archive unit test error message check. * Allows for nil output writers in snapshot RPC handlers. * Renames hash list Decode to DecodeAndVerify. * Closes the client connection for snapshot ops. * Lowers timeout for restore ops. * Updates Raft vendor to get new Restore signature and integrates with Consul. * Bounces the leader's internal state when we do a restore.	8 years ago
James Phillips	4606adae6c	Re-ups the snake oil certs for the unit tests. ... Ref #979 for a link to the blog with the commands to use :-)	9 years ago
James Phillips	794974d759	Reissues cert for the unit tests, which expired a few days ago.	10 years ago
Armon Dadgar	75d2701a1a	test: Adding hostname certs	10 years ago
Nelson Elhage	12a7f765b6	Add some basic smoke tests for wrapTLSclient. ... Check the success case, and check that we reject a self-signed certificate.	11 years ago
William Tisäter	ed4230f1fc	Fix tests on Go 1.3 and greater ... Go 1.3 and greater require ServerName or InsecureSkipVerify to be set. https://codereview.appspot.com/67010043/	11 years ago
Armon Dadgar	a4246abe7d	Adding testing certificates	11 years ago