202 Commits (8a301fb9159a4ca1baee1fb9fd9191daae82a6a5)

Author	SHA1	Message	Date
freddygv	ceb52d649a	Account for upstream targets in another DC. ... Transparent proxies typically cannot dial upstreams in remote datacenters. However, if their upstream configures a redirect to a remote DC then the upstream targets will be in another datacenter. In that sort of case we should use the WAN address for the passthrough.	3 years ago
freddygv	cbea3d203c	Fix race of upstreams with same passthrough ip ... Due to timing, a transparent proxy could have two upstreams to dial directly with the same address. For example: - The orders service can dial upstreams shipping and payment directly. - An instance of shipping at address 10.0.0.1 is deregistered. - Payments is scaled up and scheduled to have address 10.0.0.1. - The orders service receives the event for the new payments instance before seeing the deregistration for the shipping instance. At this point two upstreams have the same passthrough address and Envoy will reject the listener configuration. To disambiguate this commit considers the Raft index when storing passthrough addresses. In the example above, 10.0.0.1 would only be associated with the newer payments service instance.	3 years ago
freddygv	659ebc05a9	Ensure passthrough addresses get cleaned up ... Transparent proxies can set up filter chains that allow direct connections to upstream service instances. Services that can be dialed directly are stored in the PassthroughUpstreams map of the proxycfg snapshot. Previously these addresses were not being cleaned up based on new service health data. The list of addresses associated with an upstream service would only ever grow. As services scale up and down, eventually they will have instances assigned to an IP that was previously assigned to a different service. When IP addresses are duplicated across filter chain match rules the listener config will be rejected by Envoy. This commit updates the proxycfg snapshot management so that passthrough addresses can get cleaned up when no longer associated with a given upstream. There is still the possibility of a race condition here where due to timing an address is shared between multiple passthrough upstreams. That concern is mitigated by #12195, but will be further addressed in a follow-up.	3 years ago
freddygv	c31c1158a6	Add failing test ... The updated test fails because passthrough upstream addresses are not being cleaned up.	3 years ago
R.B. Boyer	b60d89e7ef	bulk rewrite using this script ... set -euo pipefail unset CDPATH cd "$(dirname "$0")" for f in $(git grep '\brequire := require\.New(' | cut -d':' -f1 | sort -u); do echo "=== require: $f ===" sed -i '/require := require.New(t)/d' $f # require.XXX(blah) but not require.XXX(tblah) or require.XXX(rblah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\([^tr]\)/require.\1(t,\2/g' $f # require.XXX(tblah) but not require.XXX(t, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/require.\1(t,\2/g' $f # require.XXX(rblah) but not require.XXX(r, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/require.\1(t,\2/g' $f gofmt -s -w $f done for f in $(git grep '\bassert := assert\.New(' | cut -d':' -f1 | sort -u); do echo "=== assert: $f ===" sed -i '/assert := assert.New(t)/d' $f # assert.XXX(blah) but not assert.XXX(tblah) or assert.XXX(rblah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\([^tr]\)/assert.\1(t,\2/g' $f # assert.XXX(tblah) but not assert.XXX(t, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/assert.\1(t,\2/g' $f # assert.XXX(rblah) but not assert.XXX(r, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/assert.\1(t,\2/g' $f gofmt -s -w $f done	3 years ago
R.B. Boyer	424f3cdd2c	proxycfg: introduce explicit UpstreamID in lieu of bare string (#12125) ... The gist here is that now we use a value-type struct proxycfg.UpstreamID as the map key in ConfigSnapshot maps where we used to use "upstream id-ish" strings. These are internal only and used just for bidirectional trips through the agent cache keyspace (like the discovery chain target struct). For the few places where the upstream id needs to be projected into xDS, that's what (proxycfg.UpstreamID).EnvoyID() is for. This lets us ALWAYS inject the partition and namespace into these things without making stuff like the golden testdata diverge.	3 years ago
Dhia Ayachi	e653f81919	reset `coalesceTimer` to nil as soon as the event is consumed (#11924) ... * reset `coalesceTimer` to nil as soon as the event is consumed * add change log * refactor to add relevant test. * fix linter * Apply suggestions from code review Co-authored-by: Freddy <freddygv@users.noreply.github.com> * remove non needed check Co-authored-by: Freddy <freddygv@users.noreply.github.com>	3 years ago
freddygv	21f2c2e68d	Purge chain if it shouldn't be there	3 years ago
freddygv	d26b4860fd	Account for new upstreams constraint in tests	3 years ago
freddygv	2fe27b748d	Check ingress upstreams when gating chain watches	3 years ago
freddygv	6af9a0d8cf	Avoid storing chain without an upstream	3 years ago
freddygv	ba12dc215b	Clean up chains separately from their watches	3 years ago
freddygv	70d6358426	Store intention upstreams in snapshot	3 years ago
R.B. Boyer	81ea8129d7	proxycfg: ensure all of the watches are canceled if they are cancelable (#11824)	3 years ago
R.B. Boyer	4aabbe529c	proxycfg: use external addresses in tproxy when crossing partition boundaries (#11823)	3 years ago
R.B. Boyer	631c649291	various partition related todos (#11822)	3 years ago
R.B. Boyer	1e02460bd1	re-run gofmt on 1.17 (#11579) ... This should let freshly recompiled golangci-lint binaries using Go 1.17 pass 'make lint'	3 years ago
freddygv	0e507492d0	Update proxycfg for ingress service partitions	3 years ago
Freddy	00b5b0a0a2	Update filter chain creation for sidecar/ingress listeners (#11245) ... The duo of `makeUpstreamFilterChainForDiscoveryChain` and `makeListenerForDiscoveryChain` were really hard to reason about, and led to concealing a bug in their branching logic. There were several issues here: - They tried to accomplish too much: determining filter name, cluster name, and whether RDS should be used. - They embedded logic to handle significantly different kinds of upstream listeners (passthrough, prepared query, typical services, and catch-all) - They needed to coalesce different data sources (Upstream and CompiledDiscoveryChain) Rather than handling all of those tasks inside of these functions, this PR pulls out the RDS/clusterName/filterName logic. This refactor also fixed a bug with the handling of [UpstreamDefaults](https://www.consul.io/docs/connect/config-entries/service-defaults#defaults). These defaults get stored as UpstreamConfig in the proxy snapshot with a DestinationName of "*", since they apply to all upstreams. However, this wildcard destination name must not be used when creating the name of the associated upstream cluster. The coalescing logic in the original functions here was in some situations creating clusters with a `*.` prefix, which is not a valid destination.	3 years ago
Daniel Upton	50a1f20ff9	xds: prefer fed state gateway definitions if they're fresher (#11522) ... Fixes an issue described in #10132, where if two DCs are WAN federated over mesh gateways, and the gateway in the non-primary DC is terminated and receives a new IP address (as is commonly the case when running them on ephemeral compute instances) the primary DC is unable to re-establish its connection until the agent running on its own gateway is restarted. This was happening because we always preferred gateways discovered by the `Internal.ServiceDump` RPC (which would fail because there's no way to dial the remote DC) over those discovered in the federation state, which is replicated as long as the primary DC's gateway is reachable.	3 years ago
freddygv	60066e5154	Exclude default partition from GatewayKey string ... This will behave the way we handle SNI and SPIFFE IDs, where the default partition is excluded. Excluding the default ensures that don't attempt to compare default.dc2 to dc2 in OSS.	3 years ago
freddygv	e3666b0bc4	Update GatewayKeys deduplication ... Federation states data is only keyed on datacenter, so it cannot be directly compared against keys for gateway groups.	3 years ago
freddygv	90ce897456	Store GatewayKey in proxycfg snapshot for re-use	3 years ago
freddygv	4d4ccedb3a	Update locality check in proxycfg	3 years ago
freddygv	3a2061544d	Fixup partitions assertion	3 years ago
freddygv	d28b9052b2	Move the exportingpartitions constant to enterprise	3 years ago
freddygv	448701dbd8	Replace default partition check	3 years ago
freddygv	12923f5ebc	PR comments	3 years ago
freddygv	a33b6923e0	Account for partitions in xds gen for mesh gw ... This commit avoids skipping gateways in remote partitions of the local DC when generating listeners/clusters/endpoints.	3 years ago
freddygv	110fae820a	Update xds pkg to account for GatewayKey	3 years ago
freddygv	7e65678c52	Update mesh gateway proxy watches for partitions ... This commit updates mesh gateway watches for cross-partitions communication. * Mesh gateways are keyed by partition and datacenter. * Mesh gateways will now watch gateways in partitions that export services to their partition. * Mesh gateways in non-default partitions will not have cross-datacenter watches. They are not involved in traditional WAN federation.	3 years ago
freddygv	37a16e9487	Replace Split with SplitN	3 years ago
freddygv	b9b6447977	Finish removing useInDatacenter	3 years ago
freddygv	62e0fc62c1	Configure sidecars to watch gateways in partitions ... Previously the datacenter of the gateway was the key identifier, now it is the datacenter and partition. When dialing services in other partitions or datacenters we now watch the appropriate partition.	3 years ago
Paul Banks	78a00f2e1c	Add support for enabling connect-based ingress TLS per listener.	3 years ago
Daniel Nephin	eb632c53a2	structs: rename the last helper method. ... This one gets used a bunch, but we can rename it to make the behaviour more obvious.	3 years ago
Daniel Nephin	6d72517682	structs: remove two methods that were only used once each. ... These methods only called a single function. Wrappers like this end up making code harder to read because it adds extra ways of doing things. We already have many helper functions for constructing these types, we don't need additional methods.	3 years ago
Paul Banks	136928a90f	Minor PR typo and cleanup fixes	3 years ago
Paul Banks	20d0bf81f7	Revert abandonned changes to proxycfg for Ent test consistency	3 years ago
Paul Banks	659321d008	Handle namespaces in route names correctly; add tests for enterprise	3 years ago
Paul Banks	ccbda0c285	Update proxycfg to hold more ingress config state	3 years ago
Paul Banks	4e39f03d5b	Add ingress-gateway config for SDS	3 years ago
freddygv	49248a0802	Fixup proxycfg tproxy case	3 years ago
freddygv	95a6db9cfa	Account for partitions in ixn match/decision	3 years ago
freddygv	3f3a61c6e1	Fixup manager tests	3 years ago
freddygv	77681b9f6c	Pass partition to intention match query	3 years ago
Paul Banks	e22cc9c53a	Header manip for split legs plumbing	3 years ago
Paul Banks	83fc8723a3	Header manip for service-router plumbed through	3 years ago
Paul Banks	f439dfc04f	Ingress gateway header manip plumbing	3 years ago
Dhia Ayachi	bc0e4f2f46	partition dicovery chains (#10983) ... * partition dicovery chains * fix default partition for OSS	3 years ago