73 Commits (d1bc9fbbf3d298f0f4172432dc73709c0c1e9364)

Author	SHA1	Message	Date
R.B. Boyer	201d1458c3	xds: mesh gateways now have their own leaf certificate when involved in a peering (#13460) ... This is only configured in xDS when a service with an L7 protocol is exported. They also load any relevant trust bundles for the peered services to eventually use for L7 SPIFFE validation during mTLS termination.	2 years ago
freddygv	6c8ab1bbac	Fixup stream tear-down steps. ... 1. Fix a bug where the peering leader routine would not track all active peerings in the "stored" reconciliation map. This could lead to tearing down streams where the token was generated, since the ConnectedStreams() method used for reconciliation returns all streams and not just the ones initiated by this leader routine. 2. Fix a race where stream contexts were being canceled before termination messages were being processed by a peer. Previously the leader routine would tear down streams by canceling their context right after the termination message was sent. This context cancelation could be propagated to the server side faster than the termination message. Now there is a change where the dialing peer uses CloseSend() to signal when no more messages will be sent. Eventually the server peer will read an EOF after receiving and processing the preceding termination message. Using CloseSend() is actually not enough to address the issue mentioned, since it doesn't wait for the server peer to finish processing messages. Because of this now the dialing peer also reads from the stream until an error signals that there are no more messages. Receiving an EOF from our peer indicates that they processed the termination message and have no additional work to do. Given that the stream is being closed, all the messages received by Recv are discarded. We only check for errors to avoid importing new data.	2 years ago
freddygv	cc921a9c78	Update peering state and RPC for deferred deletion ... When deleting a peering we do not want to delete the peering and all imported data in a single operation, since deleting a large amount of data at once could overload Consul. Instead we defer deletion of peerings so that: 1. When a peering deletion request is received via gRPC the peering is marked for deletion by setting the DeletedAt field. 2. A leader routine will monitor for peerings that are marked for deletion and kick off a throttled deletion of all imported resources before deleting the peering itself. This commit mostly addresses point #1 by modifying the peering service to mark peerings for deletion. Another key change is to add a PeeringListDeleted state store function which can return all peerings marked for deletion. This function is what will be watched by the deferred deletion leader routine.	2 years ago
Freddy	71b254522e	Clean up imported nodes/services/checks as needed (#13367) ... Previously, imported data would never be deleted. As nodes/services/checks were registered and deregistered, resources deleted from the exporting cluster would accumulate in the imported cluster. This commit makes updates to replication so that whenever an update is received for a service name we reconcile what was present in the catalog against what was received. This handleUpdateService method can handle both updates and deletions.	2 years ago
R.B. Boyer	7001e1151c	peering: rename initiate to establish in the context of the APIs (#13419)	3 years ago
R.B. Boyer	bba3eb8cdd	peering: mesh gateways are required for cross-peer service mesh communication (#13410) ... Require use of mesh gateways in order for service mesh data plane traffic to flow between peers. This also adds plumbing for envoy integration tests involving peers, and one starter peering test.	3 years ago
R.B. Boyer	7423886136	peering: allow protobuf requests to populate the default partition or namespace (#13398)	3 years ago
alex	bbbc50815a	peering: send leader addr (#13342) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	3 years ago
freddygv	647c57a416	Add agent cache-type for TrustBundleListByService ... There are a handful of changes in this commit: * When querying trust bundles for a service we need to be able to specify the namespace of the service. * The endpoint needs to track the index because the cache watches use it. * Extracted bulk of the endpoint's logic to a state store function so that index tracking could be tested more easily. * Removed check for service existence, deferring that sort of work to ACL authz * Added the cache type	3 years ago
Freddy	9427700270	[OSS] Add grpc endpoint to fetch a specific trust bundle (#13292) ... Co-authored-by: R.B. Boyer <rb@hashicorp.com>	3 years ago
alex	fd7a403e11	monitor leadership in peering service (#13257) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com>	3 years ago
Chris S. Kim	6d3bea7129	Add support for streaming CA roots to peers (#13260) ... Sender watches for changes to CA roots and sends them through the replication stream. Receiver saves CA roots to tablePeeringTrustBundle	3 years ago
R.B. Boyer	1a8834e1c8	peering: replicate expected SNI, SPIFFE, and service protocol to peers (#13218) ... The importing peer will need to know what SNI and SPIFFE name corresponds to each exported service. Additionally it will need to know at a high level the protocol in use (L4/L7) to generate the appropriate connection pool and local metrics. For replicated connect synthetic entities we edit the `Connect{}` part of a `NodeService` to have a new section: { "PeerMeta": { "SNI": [ "web.default.default.owt.external.183150d5-1033-3672-c426-c29205a576b8.consul" ], "SpiffeID": [ "spiffe://183150d5-1033-3672-c426-c29205a576b8.consul/ns/default/dc/dc1/svc/web" ], "Protocol": "tcp" } } This data is then replicated and saved as-is at the importing side. Both SNI and SpiffeID are slices for now until I can be sure we don't need them for how mesh gateways will ultimately work.	3 years ago
R.B. Boyer	be631ebdce	peering: disable requirement for mesh gateways initially (#13213)	3 years ago
alex	876f3bb971	peering: expose IsLeader, hung up on dialer if follower (#13164) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com> Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	3 years ago
R.B. Boyer	2e72f44fda	peering: accept replication stream of discovery chain information at the importing side (#13151)	3 years ago
R.B. Boyer	3e4a522882	peering: replicate discovery chains information to importing peers ... Treat each exported service as a "discovery chain" and replicate one synthetic CheckServiceNode for each chain and remote mesh gateway. The health will be a flattened generated check of the checks for that mesh gateway node.	3 years ago
R.B. Boyer	5a03536040	prefactor some functions out of the monolithic file	3 years ago
Freddy	b38be4c0ed	Patches to peering initiation for POC demo (#13076) ... Co-authored-by: R.B. Boyer <rb@hashicorp.com>	3 years ago
Evan Culver	0fa5e7be5a	peering: add TrustBundleListByService endpoint (#13048)	3 years ago
Freddy	4e215dc411	[OSS] Add upsert handling for receiving CheckServiceNode (#13061)	3 years ago
FFMMM	37a1e33834	expose meta tags for peering (#12964)	3 years ago
R.B. Boyer	f507f62f3c	peering: initial sync (#12842) ... - Add endpoints related to peering: read, list, generate token, initiate peering - Update node/service/check table indexing to account for peers - Foundational changes for pushing service updates to a peer - Plumb peer name through Health.ServiceNodes path see: ENT-1765, ENT-1280, ENT-1283, ENT-1283, ENT-1756, ENT-1739, ENT-1750, ENT-1679, ENT-1709, ENT-1704, ENT-1690, ENT-1689, ENT-1702, ENT-1701, ENT-1683, ENT-1663, ENT-1650, ENT-1678, ENT-1628, ENT-1658, ENT-1640, ENT-1637, ENT-1597, ENT-1634, ENT-1613, ENT-1616, ENT-1617, ENT-1591, ENT-1588, ENT-1596, ENT-1572, ENT-1555 Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: freddygv <freddy@hashicorp.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Evan Culver <eculver@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>	3 years ago