91 Commits (f05bad4a1d6cfcb2cca0ff78eb6aa19ffc9df76c)

Author	SHA1	Message	Date
R.B. Boyer	424f3cdd2c	proxycfg: introduce explicit UpstreamID in lieu of bare string (#12125) ... The gist here is that now we use a value-type struct proxycfg.UpstreamID as the map key in ConfigSnapshot maps where we used to use "upstream id-ish" strings. These are internal only and used just for bidirectional trips through the agent cache keyspace (like the discovery chain target struct). For the few places where the upstream id needs to be projected into xDS, that's what (proxycfg.UpstreamID).EnvoyID() is for. This lets us ALWAYS inject the partition and namespace into these things without making stuff like the golden testdata diverge.	3 years ago
Dhia Ayachi	e653f81919	reset `coalesceTimer` to nil as soon as the event is consumed (#11924) ... * reset `coalesceTimer` to nil as soon as the event is consumed * add change log * refactor to add relevant test. * fix linter * Apply suggestions from code review Co-authored-by: Freddy <freddygv@users.noreply.github.com> * remove non needed check Co-authored-by: Freddy <freddygv@users.noreply.github.com>	3 years ago
R.B. Boyer	631c649291	various partition related todos (#11822)	3 years ago
Freddy	00b5b0a0a2	Update filter chain creation for sidecar/ingress listeners (#11245) ... The duo of `makeUpstreamFilterChainForDiscoveryChain` and `makeListenerForDiscoveryChain` were really hard to reason about, and led to concealing a bug in their branching logic. There were several issues here: - They tried to accomplish too much: determining filter name, cluster name, and whether RDS should be used. - They embedded logic to handle significantly different kinds of upstream listeners (passthrough, prepared query, typical services, and catch-all) - They needed to coalesce different data sources (Upstream and CompiledDiscoveryChain) Rather than handling all of those tasks inside of these functions, this PR pulls out the RDS/clusterName/filterName logic. This refactor also fixed a bug with the handling of [UpstreamDefaults](https://www.consul.io/docs/connect/config-entries/service-defaults#defaults). These defaults get stored as UpstreamConfig in the proxy snapshot with a DestinationName of "*", since they apply to all upstreams. However, this wildcard destination name must not be used when creating the name of the associated upstream cluster. The coalescing logic in the original functions here was in some situations creating clusters with a `*.` prefix, which is not a valid destination.	3 years ago
freddygv	90ce897456	Store GatewayKey in proxycfg snapshot for re-use	3 years ago
freddygv	4d4ccedb3a	Update locality check in proxycfg	3 years ago
freddygv	62e0fc62c1	Configure sidecars to watch gateways in partitions ... Previously the datacenter of the gateway was the key identifier, now it is the datacenter and partition. When dialing services in other partitions or datacenters we now watch the appropriate partition.	3 years ago
Freddy	19f6e1ca31	Log the correlation ID when blocking queries fire (#10689) ... Knowing that blocking queries are firing does not provide much information on its own. If we know the correlation IDs we can piece together which parts of the snapshot have been populated. Some of these responses might be empty from the blocking query timing out. But if they're returning quickly I think we can reasonably assume they contain data.	3 years ago
Daniel Nephin	6bc5255028	proxycfg: move each handler into a seprate file ... There is no interaction between these handlers, so splitting them into separate files makes it easier to discover the full implementation of each kindHandler.	3 years ago
Daniel Nephin	19d3eeff3c	Merge pull request #9489 from hashicorp/dnephin/proxycfg-state-2 ... proxycfg: split state into a handler for each kind	4 years ago
Nitya Dhanushkodi	52043830b4	proxycfg: reference to entry in map should not panic	4 years ago
Daniel Nephin	e738fa3b80	Replace type conversion with embedded structs	4 years ago
Daniel Nephin	32c15d9a88	proxycfg: split state into kind-specific types ... This commit extracts all the kind-specific logic into handler types, and keeps the generic parts on the state struct. This change should make it easier to add new kinds, and see the implementation of each kind more clearly.	4 years ago
Daniel Nephin	cd05df7157	proxycfg: unmethod hostnameEndpoints ... the method receiver can be replaced by the first argument. This will allow us to extract more from the state struct in the future.	4 years ago
Daniel Nephin	0547d0c046	Merge pull request #9466 from hashicorp/dnephin/proxycfg-state ... proxycfg: prepare state for split by kind	4 years ago
Nitya Dhanushkodi	b8b44419a0	proxycfg: Ensure that endpoints for explicit upstreams in other datacenters are watched in transparent mode (#10391) ... Co-authored-by: Freddy Vallenilla <freddy@hashicorp.com>	4 years ago
Daniel Nephin	016c5611d1	proxycfg: extract two types from state struct ... These two new struct types will allow us to make polymorphic handler for each kind, instad of having all the logic for each proxy kind on the state struct.	4 years ago
Daniel Nephin	9c40aa729f	proxycfg: pass context around where it is needed ... context.Context should never be stored on a struct (as it says in the godoc) because it is easy to to end up with the wrong context when it is stored. Also see https://blog.golang.org/context-and-structs This change is also in preparation for splitting state into kind-specific handlers so that the implementation of each kind is grouped together.	4 years ago
Freddy	429f9d8bb8	Add flag for transparent proxies to dial individual instances (#10329)	4 years ago
Mark Anderson	06f0f79218	Continue working through proxy and agent ... Rework/listeners, rename makeListener Refactor, tests pass Signed-off-by: Mark Anderson <manderson@hashicorp.com>	4 years ago
Freddy	ed1082510d	Fixup discovery chain handling in transparent mode (#10168) ... Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> Previously we would associate the address of a discovery chain target with the discovery chain's filter chain. This was broken for a few reasons: - If the upstream is a virtual service, the client proxy has no way of dialing it because virtual services are not targets of their discovery chains. The targets are distinct services. This is addressed by watching the endpoints of all upstream services, not just their discovery chain targets. - If multiple discovery chains resolve to the same target, that would lead to multiple filter chains attempting to match on the target's virtual IP. This is addressed by only matching on the upstream's virtual IP. NOTE: this implementation requires an intention to the redirecting virtual service and not just to the final destination. This is how we can know that the virtual service is an upstream to watch. A later PR will look into traversing discovery chains when computing upstreams so that intentions are only required to the discovery chain targets.	4 years ago
Freddy	078c40425f	Rename "cluster" config entry to "mesh" (#10127) ... This config entry is being renamed primarily because in k8s the name cluster could be confusing given that the config entry applies across federated datacenters. Additionally, this config entry will only apply to Consul as a service mesh, so the more generic "cluster" name is not needed.	4 years ago
freddygv	7bd51ff536	Replace TransparentProxy bool with ProxyMode ... This PR replaces the original boolean used to configure transparent proxy mode. It was replaced with a string mode that can be set to: - "": Empty string is the default for when the setting should be defaulted from other configuration like config entries. - "direct": Direct mode is how applications originally opted into the mesh. Proxy listeners need to be dialed directly. - "transparent": Transparent mode enables configuring Envoy as a transparent proxy. Traffic must be captured and redirected to the inbound and outbound listeners. This PR also adds a struct for transparent proxy specific configuration. Initially this is not stored as a pointer. Will revisit that decision before GA.	4 years ago
freddygv	b21224a4c8	PR comments	4 years ago
freddygv	49a4a78fd5	Ensure mesh gateway mode override is set for upstreams for intentions	4 years ago
freddygv	5140c3e51f	Finish resolving upstream defaults in proxycfg	4 years ago
R.B. Boyer	499fee73b3	connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973) ... This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.	4 years ago
freddygv	eb1e0a1751	Cancel watch on all errors	4 years ago
freddygv	f4f45af6d0	Merge master and fix upstream config protocol defaulting	4 years ago
freddygv	0da8702f34	PR comments	4 years ago
freddygv	a54d6a9010	Update proxycfg for transparent proxy	4 years ago
Daniel Nephin	f40b76af2d	proxycfg: use rpcclient/health.Client instead of passing around cache name ... This should allow us to swap out the implementation with something other than `agent/cache` without making further code changes.	4 years ago
Daniel Nephin	906834ce8e	proxycfg: Use streaming in connect state	4 years ago
freddygv	ec5f75776b	Update comments on avoiding proxycfg deadlock	4 years ago
freddygv	6e443e5536	Retry send after timer fires, in case no updates occur	4 years ago
freddygv	95e7641faa	Update proxycfg logging, labels were already attached	4 years ago
freddygv	5ba14ad41d	Add trace logs to proxycfg state runner and xds srv	4 years ago
freddygv	37190c0d0d	Avoid potential deadlock using non-blocking send ... Deadlock scenario: 1. Due to scheduling, the state runner sends one snapshot into snapCh and then attempts to send a second. The first send succeeds because the channel is buffered, but the second blocks. 2. Separately, Manager.Watch is called by the xDS server after getting a discovery request from Envoy. This function acquires the manager lock and then blocks on receiving the CurrentSnapshot from the state runner. 3. Separately, there is a Manager goroutine that reads the snapshots from the channel in step 1. These reads are done to notify proxy watchers, but they require holding the manager lock. This goroutine goes to acquire that lock, but can't because it is held by step 2. Now, the goroutine from step 3 is waiting on the one from step 2 to release the lock. The goroutine from step 2 won't release the lock until the goroutine in step 1 advances. But the goroutine in step 1 is waiting for the one in step 3. Deadlock. By making this send non-blocking step 1 above can proceed. The coalesce timer will be reset and a new valid snapshot will be delivered after it elapses or when one is requested by xDS.	4 years ago
freddygv	7fd518ff1d	Merge master	4 years ago
freddygv	87541ab80a	Fix type assertion	4 years ago
R.B. Boyer	74d5df7c7a	xds: use envoy's rbac filter to handle intentions entirely within envoy (#8569)	4 years ago
Daniel Nephin	5afcf5c1bc	Merge pull request #8034 from hashicorp/dnephin/add-linter-staticcheck-4 ... ci: enable SA4006 staticcheck check and add ineffassign	5 years ago
Daniel Nephin	068b43df90	Enable gofmt simplify ... Code changes done automatically with 'gofmt -s -w'	5 years ago
Daniel Nephin	cb050b280c	ci: enable SA4006 staticcheck check ... And fix the 'value not used' issues. Many of these are not bugs, but a few are tests not checking errors, and one appears to be a missed error in non-test code.	5 years ago
freddygv	19e3954603	Move compound service names to use ServiceName type	5 years ago
Freddy	166a8b2a58	Only pass one hostname via EDS and prefer healthy ones (#8084) ... Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Currently when passing hostname clusters to Envoy, we set each service instance registered with Consul as an LbEndpoint for the cluster. However, Envoy can only handle one per cluster: [2020-06-04 18:32:34.094][1][warning][config] [source/common/config/grpc_subscription_impl.cc:87] gRPC config for type.googleapis.com/envoy.api.v2.Cluster rejected: Error adding/updating cluster(s) dc2.internal.ddd90499-9b47-91c5-4616-c0cbf0fc358a.consul: LOGICAL_DNS clusters must have a single locality_lb_endpoint and a single lb_endpoint, server.dc2.consul: LOGICAL_DNS clusters must have a single locality_lb_endpoint and a single lb_endpoint Envoy is currently handling this gracefully by only picking one of the endpoints. However, we should avoid passing multiple to avoid these warning logs. This PR: * Ensures we only pass one endpoint, which is tied to one service instance. * We prefer sending an endpoint which is marked as Healthy by Consul. * If no endpoints are healthy we emit a warning and skip the cluster. * If multiple unique hostnames are spread across service instances we emit a warning and let the user know which will be resolved.	5 years ago
Freddy	9ed325ba8b	Enable gateways to resolve hostnames to IPv4 addresses (#7999) ... The DNS resolution will be handled by Envoy and defaults to LOGICAL_DNS. This discovery type can be overridden on a per-gateway basis with the envoy_dns_discovery_type Gateway Option. If a service contains an instance with a hostname as an address we set the Envoy cluster to use DNS as the discovery type rather than EDS. Since both mesh gateways and terminating gateways route to clusters using SNI, whenever there is a mix of hostnames and IP addresses associated with a service we use the hostname + CDS rather than the IPs + EDS. Note that we detect hostnames by attempting to parse the service instance's address as an IP. If it is not a valid IP we assume it is a hostname.	5 years ago
Daniel Nephin	c88fae0aac	ci: Add staticcheck and fix most errors ... Three of the checks are temporarily disabled to limit the size of the diff, and allow us to enable all the other checks in CI. In a follow up we can fix the issues reported by the other checks one at a time, and enable them.	5 years ago
Kyle Havlovitz	b14696e32a	Standardize support for Tagged and BindAddresses in Ingress Gateways (#7924) ... * Standardize support for Tagged and BindAddresses in Ingress Gateways This updates the TaggedAddresses and BindAddresses behavior for Ingress to match Mesh/Terminating gateways. The `consul connect envoy` command now also allows passing an address without a port for tagged/bind addresses. * Update command/connect/envoy/envoy.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * PR comments * Check to see if address is an actual IP address * Update agent/xds/listeners.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * fix whitespace Co-authored-by: Chris Piraino <cpiraino@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com>	5 years ago
Chris Piraino	a0e1f57ac2	Remove development log line	5 years ago