121 Commits (9a5a0ba20965dab1595de2b8e240bed2d9596e5f)

Author	SHA1	Message	Date
R.B. Boyer	71d45a3460	Support Incremental xDS mode (#9855) ... This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged. Union of all commit messages follows to give an overarching summary: xds: exclusively support incremental xDS when using xDS v3 Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail. Work around a strange older envoy behavior involving empty CDS responses over incremental xDS. xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead. Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client. xds: pull out checkStreamACLs method in advance of a later commit xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty. This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10. xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time	4 years ago
Freddy	078c40425f	Rename "cluster" config entry to "mesh" (#10127) ... This config entry is being renamed primarily because in k8s the name cluster could be confusing given that the config entry applies across federated datacenters. Additionally, this config entry will only apply to Consul as a service mesh, so the more generic "cluster" name is not needed.	4 years ago
Daniel Nephin	2a26085b2c	connect: do not set QuerySource.Node ... Setting this field to a value is equivalent to using the 'near' query paramter. The intent is to sort the results by proximity to the node requesting them. However with connect we send the results to envoy, which doesn't care about the order, so setting this field is increasing the work performed for no gain. It is necessary to unset this field now because we would like connect to use streaming, but streaming does not support sorting by proximity.	4 years ago
Freddy	439a7fce2d	Split Upstream.Identifier() so non-empty namespace is always prepended in ent (#10031)	4 years ago
freddygv	8857195437	Fixup wildcard ent assertion	4 years ago
freddygv	7bd51ff536	Replace TransparentProxy bool with ProxyMode ... This PR replaces the original boolean used to configure transparent proxy mode. It was replaced with a string mode that can be set to: - "": Empty string is the default for when the setting should be defaulted from other configuration like config entries. - "direct": Direct mode is how applications originally opted into the mesh. Proxy listeners need to be dialed directly. - "transparent": Transparent mode enables configuring Envoy as a transparent proxy. Traffic must be captured and redirected to the inbound and outbound listeners. This PR also adds a struct for transparent proxy specific configuration. Initially this is not stored as a pointer. Will revisit that decision before GA.	4 years ago
freddygv	b21224a4c8	PR comments	4 years ago
freddygv	49a4a78fd5	Ensure mesh gateway mode override is set for upstreams for intentions	4 years ago
freddygv	5140c3e51f	Finish resolving upstream defaults in proxycfg	4 years ago
R.B. Boyer	499fee73b3	connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973) ... This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.	4 years ago
freddygv	098b9af901	Fixup enterprise tests from tproxy changes	4 years ago
freddygv	eb1e0a1751	Cancel watch on all errors	4 years ago
freddygv	f4f45af6d0	Merge master and fix upstream config protocol defaulting	4 years ago
freddygv	0da8702f34	PR comments	4 years ago
freddygv	a54d6a9010	Update proxycfg for transparent proxy	4 years ago
Daniel Nephin	f40b76af2d	proxycfg: use rpcclient/health.Client instead of passing around cache name ... This should allow us to swap out the implementation with something other than `agent/cache` without making further code changes.	4 years ago
Daniel Nephin	906834ce8e	proxycfg: Use streaming in connect state	4 years ago
Freddy	82c269a7c5	Avoid potential proxycfg/xDS deadlock using non-blocking send	4 years ago
freddygv	ec5f75776b	Update comments on avoiding proxycfg deadlock	4 years ago
R.B. Boyer	43193a35c6	xds: prevent LDS flaps in mesh gateways due to unstable datacenter lists (#9651) ... Also fix a similar issue in Terminating Gateways that was masked by an overzealous test.	4 years ago
freddygv	6e443e5536	Retry send after timer fires, in case no updates occur	4 years ago
freddygv	95e7641faa	Update proxycfg logging, labels were already attached	4 years ago
freddygv	5ba14ad41d	Add trace logs to proxycfg state runner and xds srv	4 years ago
freddygv	37190c0d0d	Avoid potential deadlock using non-blocking send ... Deadlock scenario: 1. Due to scheduling, the state runner sends one snapshot into snapCh and then attempts to send a second. The first send succeeds because the channel is buffered, but the second blocks. 2. Separately, Manager.Watch is called by the xDS server after getting a discovery request from Envoy. This function acquires the manager lock and then blocks on receiving the CurrentSnapshot from the state runner. 3. Separately, there is a Manager goroutine that reads the snapshots from the channel in step 1. These reads are done to notify proxy watchers, but they require holding the manager lock. This goroutine goes to acquire that lock, but can't because it is held by step 2. Now, the goroutine from step 3 is waiting on the one from step 2 to release the lock. The goroutine from step 2 won't release the lock until the goroutine in step 1 advances. But the goroutine in step 1 is waiting for the one in step 3. Deadlock. By making this send non-blocking step 1 above can proceed. The coalesce timer will be reset and a new valid snapshot will be delivered after it elapses or when one is requested by xDS.	4 years ago
Daniel Nephin	b9e60c0775	testing: skip slow tests with -short ... Add a skip condition to all tests slower than 100ms. This change was made using `gotestsum tool slowest` with data from the last 3 CI runs of master. See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests With this change: ``` $ time go test -count=1 -short ./agent ok github.com/hashicorp/consul/agent 0.743s real 0m4.791s $ time go test -count=1 -short ./agent/consul ok github.com/hashicorp/consul/agent/consul 4.229s real 0m8.769s ```	4 years ago
freddygv	856d5a25ee	Fix text type assertion	4 years ago
freddygv	7fd518ff1d	Merge master	4 years ago
freddygv	87541ab80a	Fix type assertion	4 years ago
freddygv	768dbaa68d	Add session flag to cookie config	4 years ago
freddygv	eab90ea9fa	Revert EnvoyConfig nesting	4 years ago
freddygv	30ba080d25	Add explicit protocol overrides in tgw xds test cases	4 years ago
freddygv	f81fe6a1a1	Remove LB infix and move injection to xds	4 years ago
freddygv	63f79e5f9b	Restructure structs and other PR comments	4 years ago
freddygv	28d0602fc1	Pass LB config to Envoy via xDS	4 years ago
R.B. Boyer	74d5df7c7a	xds: use envoy's rbac filter to handle intentions entirely within envoy (#8569)	4 years ago
Matt Keeler	be01c4241d	Default Cache rate limiting options in New ... Also get rid of the TestCache helper which was where these defaults were happening previously.	4 years ago
Pierre Souchay	505de6dc29	Added ratelimit to handle throtling cache (#8226) ... This implements a solution for #7863 It does: Add a new config cache.entry_fetch_rate to limit the number of calls/s for a given cache entry, default value = rate.Inf Add cache.entry_fetch_max_burst size of rate limit (default value = 2) The new configuration now supports the following syntax for instance to allow 1 query every 3s: command line HCL: -hcl 'cache = { entry_fetch_rate = 0.333}' in JSON { "cache": { "entry_fetch_rate": 0.333 } }	4 years ago
Matt Keeler	12acdd7481	Disable background cache refresh for Connect Leaf Certs ... The rationale behind removing them is that all of our own code (xDS, builtin connect proxy) use the cache notification mechanism. This ensures that the blocking fetch behind the scenes is always executing. Therefore the only way you might go to get a certificate and have to wait is when 1) the request has never been made for that cert before or 2) you are using the v1/agent/connect/ca/leaf API for retrieving the cert yourself. In the first case, the refresh change doesn’t alter the behavior. In the second case, it can be mitigated by using blocking queries with that API which just like normal cache notification mechanism will cause the blocking fetch to be initiated and to get leaf certs as soon as needed. If you are not using blocking queries, or Envoy/xDS, or the builtin connect proxy but are retrieving the certs yourself then the HTTP endpoint might take a little longer to respond. This also renames the RefreshTimeout field on the register options to QueryTimeout to more accurately reflect that it is used for any type that supports blocking queries.	4 years ago
Daniel Nephin	010a609912	Fix a bunch of unparam lint issues	4 years ago
Freddy	5baa7b1b04	Always return a gateway cluster (#8158)	5 years ago
Daniel Nephin	5afcf5c1bc	Merge pull request #8034 from hashicorp/dnephin/add-linter-staticcheck-4 ... ci: enable SA4006 staticcheck check and add ineffassign	5 years ago
Daniel Nephin	068b43df90	Enable gofmt simplify ... Code changes done automatically with 'gofmt -s -w'	5 years ago
Daniel Nephin	cb050b280c	ci: enable SA4006 staticcheck check ... And fix the 'value not used' issues. Many of these are not bugs, but a few are tests not checking errors, and one appears to be a missed error in non-test code.	5 years ago
freddygv	19e3954603	Move compound service names to use ServiceName type	5 years ago
Freddy	166a8b2a58	Only pass one hostname via EDS and prefer healthy ones (#8084) ... Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Currently when passing hostname clusters to Envoy, we set each service instance registered with Consul as an LbEndpoint for the cluster. However, Envoy can only handle one per cluster: [2020-06-04 18:32:34.094][1][warning][config] [source/common/config/grpc_subscription_impl.cc:87] gRPC config for type.googleapis.com/envoy.api.v2.Cluster rejected: Error adding/updating cluster(s) dc2.internal.ddd90499-9b47-91c5-4616-c0cbf0fc358a.consul: LOGICAL_DNS clusters must have a single locality_lb_endpoint and a single lb_endpoint, server.dc2.consul: LOGICAL_DNS clusters must have a single locality_lb_endpoint and a single lb_endpoint Envoy is currently handling this gracefully by only picking one of the endpoints. However, we should avoid passing multiple to avoid these warning logs. This PR: * Ensures we only pass one endpoint, which is tied to one service instance. * We prefer sending an endpoint which is marked as Healthy by Consul. * If no endpoints are healthy we emit a warning and skip the cluster. * If multiple unique hostnames are spread across service instances we emit a warning and let the user know which will be resolved.	5 years ago
Freddy	9ed325ba8b	Enable gateways to resolve hostnames to IPv4 addresses (#7999) ... The DNS resolution will be handled by Envoy and defaults to LOGICAL_DNS. This discovery type can be overridden on a per-gateway basis with the envoy_dns_discovery_type Gateway Option. If a service contains an instance with a hostname as an address we set the Envoy cluster to use DNS as the discovery type rather than EDS. Since both mesh gateways and terminating gateways route to clusters using SNI, whenever there is a mix of hostnames and IP addresses associated with a service we use the hostname + CDS rather than the IPs + EDS. Note that we detect hostnames by attempting to parse the service instance's address as an IP. If it is not a valid IP we assume it is a hostname.	5 years ago
Daniel Nephin	c88fae0aac	ci: Add staticcheck and fix most errors ... Three of the checks are temporarily disabled to limit the size of the diff, and allow us to enable all the other checks in CI. In a follow up we can fix the issues reported by the other checks one at a time, and enable them.	5 years ago
Kyle Havlovitz	b14696e32a	Standardize support for Tagged and BindAddresses in Ingress Gateways (#7924) ... * Standardize support for Tagged and BindAddresses in Ingress Gateways This updates the TaggedAddresses and BindAddresses behavior for Ingress to match Mesh/Terminating gateways. The `consul connect envoy` command now also allows passing an address without a port for tagged/bind addresses. * Update command/connect/envoy/envoy.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * PR comments * Check to see if address is an actual IP address * Update agent/xds/listeners.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * fix whitespace Co-authored-by: Chris Piraino <cpiraino@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com>	5 years ago
Chris Piraino	9d9e23cc44	Add service id context to the proxycfg logger ... This is especially useful when multiple proxies are all querying the same Consul agent.	5 years ago
Kyle Havlovitz	136549205c	Merge pull request #7759 from hashicorp/ingress/tls-hosts ... Add TLS option for Ingress Gateway listeners	5 years ago