consul

Commit Graph

Author	SHA1	Message	Date
Chris S. Kim	fb5eb20563	Pass trust domain to RBAC to validate and fix use of wrong peer trust bundles (#13508 )	2 years ago
DanStough	4b402e3119	feat: tgtwy xDS generation for destinations Signed-off-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
R.B. Boyer	201d1458c3	xds: mesh gateways now have their own leaf certificate when involved in a peering (#13460 ) This is only configured in xDS when a service with an L7 protocol is exported. They also load any relevant trust bundles for the peered services to eventually use for L7 SPIFFE validation during mTLS termination.	2 years ago
R.B. Boyer	f557509e58	xds: allow for peered upstreams to use tagged addresses that are hostnames (#13422 ) Mesh gateways can use hostnames in their tagged addresses (#7999). This is useful if you were to expose a mesh gateway using a cloud networking load balancer appliance that gives you a DNS name but no reliable static IPs. Envoy cannot accept hostnames via EDS and those must be configured using CDS. There was already logic when configuring gateways in other locations in the code, but given the illusions in play for peering the downstream of a peered service wasn't aware that it should be doing that. Also: - ensuring that we always try to use wan-like addresses to cross peer boundaries.	2 years ago
R.B. Boyer	ab758b7b32	peering: allow mesh gateways to proxy L4 peered traffic (#13339 ) Mesh gateways will now enable tcp connections with SNI names including peering information so that those connections may be proxied. Note: this does not change the callers to use these mesh gateways.	2 years ago
Dan Upton	b168424398	xds: remove HTTPCheckFetcher dependency (#13366 ) This is the OSS portion of enterprise PR 1994 Rather than directly interrogating the agent-local state for HTTP checks using the `HTTPCheckFetcher` interface, we now rely on the config snapshot containing the checks. This reduces the number of changes required to support server xDS sessions. It's not clear why the fetching approach was introduced in `931d167ebb`.	2 years ago
R.B. Boyer	019aeaa57d	peering: update how cross-peer upstreams and represented in proxycfg and rendered in xds (#13362 ) This removes unnecessary, vestigal remnants of discovery chains.	3 years ago
Freddy	a09c776645	Update public listener with SPIFFE Validator Envoy's SPIFFE certificate validation extension allows for us to validate against different root certificates depending on the trust domain of the dialing proxy. If there are any trust bundles from peers in the config snapshot then we use the SPIFFE validator as the validation context, rather than the usual TrustedCA. The injected validation config includes the local root certificates as well.	3 years ago
Freddy	74ca6406ea	Configure upstream TLS context with peer root certs (#13321 ) For mTLS to work between two proxies in peered clusters with different root CAs, proxies need to configure their outbound listener to use different root certificates for validation. Up until peering was introduced proxies would only ever use one set of root certificates to validate all mesh traffic, both inbound and outbound. Now an upstream proxy may have a leaf certificate signed by a CA that's different from the dialing proxy's. This PR makes changes to proxycfg and xds so that the upstream TLS validation uses different root certificates depending on which cluster is being dialed.	3 years ago
Dan Upton	adeabed126	proxycfg: replace direct agent cache usage with interfaces (#13320 ) This is the OSS portion of enterprise PRs 1904, 1905, 1906, 1907, 1949, and 1971. It replaces the proxycfg manager's direct dependency on the agent cache with interfaces that will be implemented differently when serving xDS sessions from a Consul server.	3 years ago
freddygv	364758ef2f	Use embedded SpiffeID for peered upstreams	3 years ago
Dan Upton	2427e38839	Enable servers to configure arbitrary proxies from the catalog (#13244 ) OSS port of enterprise PR 1822 Includes the necessary changes to the `proxycfg` and `xds` packages to enable Consul servers to configure arbitrary proxies using catalog data. Broadly, `proxycfg.Manager` now has public methods for registering, deregistering, and listing registered proxies — the existing local agent state-sync behavior has been moved into a separate component that makes use of these methods. When an xDS session is started for a proxy service in the catalog, a goroutine will be spawned to watch the service in the server's state store and re-register it with the `proxycfg.Manager` whenever it is updated (and clean it up when the client goes away).	3 years ago
DanStough	2e2c71d2f2	fix: multiple grpc/http2 services for ingress listeners	3 years ago
Dan Upton	d7f8a8e4ef	proxycfg: remove dependency on `cache.UpdateEvent` (#13144 ) OSS portion of enterprise PR 1857. This removes (most) references to the `cache.UpdateEvent` type in the `proxycfg` package. As we're going to be direct usage of the agent cache with interfaces that can be satisfied by alternative server-local datasources, it doesn't make sense to depend on this type everywhere anymore (particularly on the `state.ch` channel). We also plan to extract `proxycfg` out of Consul into a shared library in the future, which would require removing this dependency. Aside from a fairly rote find-and-replace, the main change is that the `cache.Cache` and `health.Client` types now accept a callback function parameter, rather than a `chan<- cache.UpdateEvents`. This allows us to do the type conversion without running another goroutine.	3 years ago
Matt Keeler	b788691fa6	Watch the singular service resolver instead of the list + filtering to 1 (#13012 ) * Watch the singular service resolver instead of the list + filtering to 1 * Rename the ConfigEntries cache type to ConfigEntryList	3 years ago
Chris S. Kim	abc472f2a3	Default discovery chain when upstream targets a DestinationPeer (#12942 )	3 years ago
Chris S. Kim	9791bad136	peering: Make Upstream peer-aware (#12900 ) Adds DestinationPeer field to Upstream. Adds Peer field to UpstreamID and its string conversion functions.	3 years ago
Eric	b01bb41553	Implement routing and intentions for AWS Lambdas	3 years ago
R.B. Boyer	25ba9c147a	xds: ensure that all connect timeout configs can apply equally to tproxy direct dial connections (#12711 ) Just like standard upstreams the order of applicability in descending precedence: 1. caller's `service-defaults` upstream override for destination 2. caller's `service-defaults` upstream defaults 3. destination's `service-resolver` ConnectTimeout 4. system default of 5s Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>	3 years ago
Mark Anderson	98a2e282be	Fixup acl.EnterpriseMeta Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Kyle Havlovitz	1a3b885027	Use the GatewayService SNI field for upstream SAN validation	3 years ago
Eric	e0a15690ae	Implement Lambda Patching in the Serverless Plugin	3 years ago
R.B. Boyer	e79ce8ab03	xds: adding control of the mesh-wide min/max TLS versions and cipher suites from the mesh config entry (#12601 ) - `tls.incoming`: applies to the inbound mTLS targeting the public listener on `connect-proxy` and `terminating-gateway` envoy instances - `tls.outgoing`: applies to the outbound mTLS dialing upstreams from `connect-proxy` and `ingress-gateway` envoy instances Fixes #11966	3 years ago
R.B. Boyer	ac5bea862a	server: ensure that service-defaults meta is incorporated into the discovery chain response (#12511 ) Also add a new "Default" field to the discovery chain response to clients	3 years ago
Eric	cf3e517d0e	Create and wire up the serverless patcher	3 years ago
R.B. Boyer	2a56e0055b	proxycfg: change how various proxycfg test helpers for making ConfigSnapshot copies works to be more correct and less error prone (#12531 ) Prior to this PR for the envoy xDS golden tests in the agent/xds package we were hand-creating a proxycfg.ConfigSnapshot structure in the proper format for input to the xDS generator. Over time this intermediate structure has gotten trickier to build correctly for the various tests. This PR proposes to switch to using the existing mechanism for turning a structs.NodeService and a sequence of cache.UpdateEvent copies into a proxycfg.ConfigSnapshot, as that is less error prone to construct and aligns more with how the data arrives. NOTE: almost all of this is in test-related code. I tried super hard to craft correct event inputs to get the golden files to be the same, or similar enough after construction to feel ok that i recreated the spirit of the original test cases.	3 years ago
freddygv	ceb52d649a	Account for upstream targets in another DC. Transparent proxies typically cannot dial upstreams in remote datacenters. However, if their upstream configures a redirect to a remote DC then the upstream targets will be in another datacenter. In that sort of case we should use the WAN address for the passthrough.	3 years ago
freddygv	cbea3d203c	Fix race of upstreams with same passthrough ip Due to timing, a transparent proxy could have two upstreams to dial directly with the same address. For example: - The orders service can dial upstreams shipping and payment directly. - An instance of shipping at address 10.0.0.1 is deregistered. - Payments is scaled up and scheduled to have address 10.0.0.1. - The orders service receives the event for the new payments instance before seeing the deregistration for the shipping instance. At this point two upstreams have the same passthrough address and Envoy will reject the listener configuration. To disambiguate this commit considers the Raft index when storing passthrough addresses. In the example above, 10.0.0.1 would only be associated with the newer payments service instance.	3 years ago
freddygv	659ebc05a9	Ensure passthrough addresses get cleaned up Transparent proxies can set up filter chains that allow direct connections to upstream service instances. Services that can be dialed directly are stored in the PassthroughUpstreams map of the proxycfg snapshot. Previously these addresses were not being cleaned up based on new service health data. The list of addresses associated with an upstream service would only ever grow. As services scale up and down, eventually they will have instances assigned to an IP that was previously assigned to a different service. When IP addresses are duplicated across filter chain match rules the listener config will be rejected by Envoy. This commit updates the proxycfg snapshot management so that passthrough addresses can get cleaned up when no longer associated with a given upstream. There is still the possibility of a race condition here where due to timing an address is shared between multiple passthrough upstreams. That concern is mitigated by #12195, but will be further addressed in a follow-up.	3 years ago
freddygv	c31c1158a6	Add failing test The updated test fails because passthrough upstream addresses are not being cleaned up.	3 years ago
R.B. Boyer	b60d89e7ef	bulk rewrite using this script set -euo pipefail unset CDPATH cd "$(dirname "$0")" for f in $(git grep '\brequire := require\.New(' \| cut -d':' -f1 \| sort -u); do echo "=== require: $f ===" sed -i '/require := require.New(t)/d' $f # require.XXX(blah) but not require.XXX(tblah) or require.XXX(rblah) sed -i 's/\brequire\.$[a-zA-Z0-9_]$($[^tr]$/require.\1(t,\2/g' $f # require.XXX(tblah) but not require.XXX(t, blah) sed -i 's/\brequire\.$[a-zA-Z0-9_]$($t[^,]$/require.\1(t,\2/g' $f # require.XXX(rblah) but not require.XXX(r, blah) sed -i 's/\brequire\.$[a-zA-Z0-9_]$($r[^,]$/require.\1(t,\2/g' $f gofmt -s -w $f done for f in $(git grep '\bassert := assert\.New(' \| cut -d':' -f1 \| sort -u); do echo "=== assert: $f ===" sed -i '/assert := assert.New(t)/d' $f # assert.XXX(blah) but not assert.XXX(tblah) or assert.XXX(rblah) sed -i 's/\bassert\.$[a-zA-Z0-9_]$($[^tr]$/assert.\1(t,\2/g' $f # assert.XXX(tblah) but not assert.XXX(t, blah) sed -i 's/\bassert\.$[a-zA-Z0-9_]$($t[^,]$/assert.\1(t,\2/g' $f # assert.XXX(rblah) but not assert.XXX(r, blah) sed -i 's/\bassert\.$[a-zA-Z0-9_]$($r[^,]$/assert.\1(t,\2/g' $f gofmt -s -w $f done	3 years ago
R.B. Boyer	424f3cdd2c	proxycfg: introduce explicit UpstreamID in lieu of bare string (#12125 ) The gist here is that now we use a value-type struct proxycfg.UpstreamID as the map key in ConfigSnapshot maps where we used to use "upstream id-ish" strings. These are internal only and used just for bidirectional trips through the agent cache keyspace (like the discovery chain target struct). For the few places where the upstream id needs to be projected into xDS, that's what (proxycfg.UpstreamID).EnvoyID() is for. This lets us ALWAYS inject the partition and namespace into these things without making stuff like the golden testdata diverge.	3 years ago
Dhia Ayachi	e653f81919	reset `coalesceTimer` to nil as soon as the event is consumed (#11924 ) * reset `coalesceTimer` to nil as soon as the event is consumed * add change log * refactor to add relevant test. * fix linter * Apply suggestions from code review Co-authored-by: Freddy <freddygv@users.noreply.github.com> * remove non needed check Co-authored-by: Freddy <freddygv@users.noreply.github.com>	3 years ago
freddygv	21f2c2e68d	Purge chain if it shouldn't be there	3 years ago
freddygv	d26b4860fd	Account for new upstreams constraint in tests	3 years ago
freddygv	2fe27b748d	Check ingress upstreams when gating chain watches	3 years ago
freddygv	6af9a0d8cf	Avoid storing chain without an upstream	3 years ago
freddygv	ba12dc215b	Clean up chains separately from their watches	3 years ago
freddygv	70d6358426	Store intention upstreams in snapshot	3 years ago
R.B. Boyer	81ea8129d7	proxycfg: ensure all of the watches are canceled if they are cancelable (#11824 )	3 years ago
R.B. Boyer	4aabbe529c	proxycfg: use external addresses in tproxy when crossing partition boundaries (#11823 )	3 years ago
R.B. Boyer	631c649291	various partition related todos (#11822 )	3 years ago
R.B. Boyer	1e02460bd1	re-run gofmt on 1.17 (#11579 ) This should let freshly recompiled golangci-lint binaries using Go 1.17 pass 'make lint'	3 years ago
freddygv	0e507492d0	Update proxycfg for ingress service partitions	3 years ago
Freddy	00b5b0a0a2	Update filter chain creation for sidecar/ingress listeners (#11245 ) The duo of `makeUpstreamFilterChainForDiscoveryChain` and `makeListenerForDiscoveryChain` were really hard to reason about, and led to concealing a bug in their branching logic. There were several issues here: - They tried to accomplish too much: determining filter name, cluster name, and whether RDS should be used. - They embedded logic to handle significantly different kinds of upstream listeners (passthrough, prepared query, typical services, and catch-all) - They needed to coalesce different data sources (Upstream and CompiledDiscoveryChain) Rather than handling all of those tasks inside of these functions, this PR pulls out the RDS/clusterName/filterName logic. This refactor also fixed a bug with the handling of [UpstreamDefaults](https://www.consul.io/docs/connect/config-entries/service-defaults#defaults). These defaults get stored as UpstreamConfig in the proxy snapshot with a DestinationName of "", since they apply to all upstreams. However, this wildcard destination name must not be used when creating the name of the associated upstream cluster. The coalescing logic in the original functions here was in some situations creating clusters with a `.` prefix, which is not a valid destination.	3 years ago
Daniel Upton	50a1f20ff9	xds: prefer fed state gateway definitions if they're fresher (#11522 ) Fixes an issue described in #10132, where if two DCs are WAN federated over mesh gateways, and the gateway in the non-primary DC is terminated and receives a new IP address (as is commonly the case when running them on ephemeral compute instances) the primary DC is unable to re-establish its connection until the agent running on its own gateway is restarted. This was happening because we always preferred gateways discovered by the `Internal.ServiceDump` RPC (which would fail because there's no way to dial the remote DC) over those discovered in the federation state, which is replicated as long as the primary DC's gateway is reachable.	3 years ago
freddygv	60066e5154	Exclude default partition from GatewayKey string This will behave the way we handle SNI and SPIFFE IDs, where the default partition is excluded. Excluding the default ensures that don't attempt to compare default.dc2 to dc2 in OSS.	3 years ago
freddygv	e3666b0bc4	Update GatewayKeys deduplication Federation states data is only keyed on datacenter, so it cannot be directly compared against keys for gateway groups.	3 years ago
freddygv	90ce897456	Store GatewayKey in proxycfg snapshot for re-use	3 years ago
freddygv	4d4ccedb3a	Update locality check in proxycfg	3 years ago

1 2 3 4 5

229 Commits (CSLC-91-egress-connect-proxy)