172 Commits (d5c622246a614518d63d1f5cea4d7d69de9cda5d)

Author	SHA1	Message	Date
Chris S. Kim	a02e9abcc1	Update RBAC to handle imported services (#13404) ... When converting from Consul intentions to xds RBAC rules, services imported from other peers must encode additional data like partition (from the remote cluster) and trust domain. This PR updates the PeeringTrustBundle to hold the sending side's local partition as ExportedPartition. It also updates RBAC code to encode SpiffeIDs of imported services with the ExportedPartition and TrustDomain.	3 years ago
R.B. Boyer	7001e1151c	peering: rename initiate to establish in the context of the APIs (#13419)	3 years ago
R.B. Boyer	bba3eb8cdd	peering: mesh gateways are required for cross-peer service mesh communication (#13410) ... Require use of mesh gateways in order for service mesh data plane traffic to flow between peers. This also adds plumbing for envoy integration tests involving peers, and one starter peering test.	3 years ago
R.B. Boyer	7423886136	peering: allow protobuf requests to populate the default partition or namespace (#13398)	3 years ago
R.B. Boyer	ab758b7b32	peering: allow mesh gateways to proxy L4 peered traffic (#13339) ... Mesh gateways will now enable tcp connections with SNI names including peering information so that those connections may be proxied. Note: this does not change the callers to use these mesh gateways.	3 years ago
alex	bbbc50815a	peering: send leader addr (#13342) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	3 years ago
R.B. Boyer	019aeaa57d	peering: update how cross-peer upstreams and represented in proxycfg and rendered in xds (#13362) ... This removes unnecessary, vestigal remnants of discovery chains.	3 years ago
freddygv	647c57a416	Add agent cache-type for TrustBundleListByService ... There are a handful of changes in this commit: * When querying trust bundles for a service we need to be able to specify the namespace of the service. * The endpoint needs to track the index because the cache watches use it. * Extracted bulk of the endpoint's logic to a state store function so that index tracking could be tested more easily. * Removed check for service existence, deferring that sort of work to ACL authz * Added the cache type	3 years ago
freddygv	8b58fa8afe	Update assumptions around exported-service config ... Given that the exported-services config entry can use wildcards, the precedence for wildcards is handled as with intentions. The most exact match is the match that applies for any given service. We do not take the union of all that apply. Another update that was made was to reflect that only one exported-services config entry applies to any given service in a partition. This is a pre-existing constraint that gets enforced by the Normalize() method on that config entry type.	3 years ago
freddygv	870e7c72d7	Return SPIFFE ID for connect proxies in PeerMeta ... Proxies dialing exporting services need to know the SPIFFE ID of services dialed so that the upstream's SANs can be validated. This commit attaches the SPIFFE ID to all connect proxies exported over the peering stream so that they are available to importing clusters. The data in the SPIFFE ID cannot be re-constructed in peer clusters because the partition of exported services is overwritten on imports.	3 years ago
Freddy	9427700270	[OSS] Add grpc endpoint to fetch a specific trust bundle (#13292) ... Co-authored-by: R.B. Boyer <rb@hashicorp.com>	3 years ago
alex	fd7a403e11	monitor leadership in peering service (#13257) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com>	3 years ago
Chris S. Kim	6d3bea7129	Add support for streaming CA roots to peers (#13260) ... Sender watches for changes to CA roots and sends them through the replication stream. Receiver saves CA roots to tablePeeringTrustBundle	3 years ago
R.B. Boyer	1a8834e1c8	peering: replicate expected SNI, SPIFFE, and service protocol to peers (#13218) ... The importing peer will need to know what SNI and SPIFFE name corresponds to each exported service. Additionally it will need to know at a high level the protocol in use (L4/L7) to generate the appropriate connection pool and local metrics. For replicated connect synthetic entities we edit the `Connect{}` part of a `NodeService` to have a new section: { "PeerMeta": { "SNI": [ "web.default.default.owt.external.183150d5-1033-3672-c426-c29205a576b8.consul" ], "SpiffeID": [ "spiffe://183150d5-1033-3672-c426-c29205a576b8.consul/ns/default/dc/dc1/svc/web" ], "Protocol": "tcp" } } This data is then replicated and saved as-is at the importing side. Both SNI and SpiffeID are slices for now until I can be sure we don't need them for how mesh gateways will ultimately work.	3 years ago
R.B. Boyer	be631ebdce	peering: disable requirement for mesh gateways initially (#13213)	3 years ago
alex	876f3bb971	peering: expose IsLeader, hung up on dialer if follower (#13164) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com> Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	3 years ago
R.B. Boyer	2e72f44fda	peering: accept replication stream of discovery chain information at the importing side (#13151)	3 years ago
R.B. Boyer	3e4a522882	peering: replicate discovery chains information to importing peers ... Treat each exported service as a "discovery chain" and replicate one synthetic CheckServiceNode for each chain and remote mesh gateway. The health will be a flattened generated check of the checks for that mesh gateway node.	3 years ago
R.B. Boyer	5a03536040	prefactor some functions out of the monolithic file	3 years ago
Freddy	b38be4c0ed	Patches to peering initiation for POC demo (#13076) ... Co-authored-by: R.B. Boyer <rb@hashicorp.com>	3 years ago
Freddy	e874b860c0	Actually block when syncing subscriptions (#13066) ... By changing to use WatchCtx we will actually block for changes to the peering list. WatchCh creates a goroutine to collect errors from WatchCtx and returns immediately. The existing behavior wouldn't result in a tight loop because of the rate limiting in the surrounding function, but it would still lead to more work than is necessary.	3 years ago
Evan Culver	0fa5e7be5a	peering: add TrustBundleListByService endpoint (#13048)	3 years ago
Freddy	4e215dc411	[OSS] Add upsert handling for receiving CheckServiceNode (#13061)	3 years ago
R.B. Boyer	cc15a11f9c	test: ensure this package uses freeport for port allocation (#13036)	3 years ago
R.B. Boyer	901fd4dd68	remove remaining shim runStep functions (#13015) ... Wraps up the refactor from #13013	3 years ago
R.B. Boyer	0d6d16ddfb	add general runstep test helper instead of copying it all over the place (#13013)	3 years ago
FFMMM	37a1e33834	expose meta tags for peering (#12964)	3 years ago
R.B. Boyer	f507f62f3c	peering: initial sync (#12842) ... - Add endpoints related to peering: read, list, generate token, initiate peering - Update node/service/check table indexing to account for peers - Foundational changes for pushing service updates to a peer - Plumb peer name through Health.ServiceNodes path see: ENT-1765, ENT-1280, ENT-1283, ENT-1283, ENT-1756, ENT-1739, ENT-1750, ENT-1679, ENT-1709, ENT-1704, ENT-1690, ENT-1689, ENT-1702, ENT-1701, ENT-1683, ENT-1663, ENT-1650, ENT-1678, ENT-1628, ENT-1658, ENT-1640, ENT-1637, ENT-1597, ENT-1634, ENT-1613, ENT-1616, ENT-1617, ENT-1591, ENT-1588, ENT-1596, ENT-1572, ENT-1555 Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: freddygv <freddy@hashicorp.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Evan Culver <eculver@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>	3 years ago
FFMMM	a46bbe892d	add more labels to RequestRecorder (#12727) ... Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>	3 years ago
FFMMM	5245251bbf	[rpc/middleware][consul] plumb intercept off, add server level happy test (#12692)	3 years ago
FFMMM	7ed356b338	lower log to trace (#12708)	3 years ago
FFMMM	1adfd7b94c	polish rpc.service.call metric behavior (#12624)	3 years ago
FFMMM	c39854de78	fix bad oss sync, use gauges not counters (#12611)	3 years ago
FFMMM	a7e5ee005a	factor out recording func, add unit tests (#12585) ... Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>	3 years ago
Dan Upton	7298967070	Restructure gRPC server setup (#12586) ... OSS sync of enterprise changes at 0b44395e	3 years ago
FFMMM	e5ebc47a94	pre register new rpc metric, rename metric (#12582)	3 years ago
FFMMM	db27ea3484	[sync oss] add net/rpc interceptor implementation (#12573) ... * sync ent changes from 866dcb0667 Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> * update oss go.mod Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>	3 years ago
Dan Upton	fdfe079674	streaming: split event buffer by key (#12080)	3 years ago
Giulio Micheloni	af7b7b5693	Merge branch 'main' into serve-panic-recovery	3 years ago
Daniel Nephin	8ba760a2fc	acl: remove id and revision from Policy constructors ... The fields were removed in a previous commit. Also remove an unused constructor for PolicyMerger	3 years ago
Daniel Nephin	aea4cc5a6d	acl: remove legacy arg to store.ACLTokenSet ... And remove the tests for legacy=true	3 years ago
Giulio Micheloni	0c78ddacde	Merge branch 'main' of https://github.com/hashicorp/consul into hashicorp-main	3 years ago
R.B. Boyer	706fc8bcd0	grpc: strip local ACL tokens from RPCs during forwarding if crossing datacenters (#11099) ... Fixes #11086	3 years ago
Giulio Micheloni	655da1fc42	Merge branch 'main' into serve-panic-recovery	3 years ago
Giulio Micheloni	4b0eaa4bff	grpc, xds: recovery middleware to return and log error in case of panic ... 1) xds and grpc servers: 1.1) to use recovery middleware with callback that prints stack trace to log 1.2) callback turn the panic into a core.Internal error 2) added unit test for grpc server	3 years ago
R.B. Boyer	097e1645e3	agent: ensure that most agent behavior correctly respects partition configuration (#10880)	3 years ago
R.B. Boyer	310e775a8a	state: partition nodes and coordinates in the state store (#10859) ... Additionally: - partitioned the catalog indexes appropriately for partitioning - removed a stray reference to a non-existent index named "node.checks"	3 years ago
Daniel Nephin	f497d5ab30	acl: remove many instances of authz == nil	3 years ago
R.B. Boyer	fc9b1a277d	sync changes to oss files made in enterprise (#10670)	3 years ago
R.B. Boyer	188e8dc51f	agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning (#10669)	3 years ago
Daniel Nephin	71b0f0a7a6	structs: remove EnterpriseMeta.GetNamespace ... I added this recently without realizing that the method already existed and was named NamespaceOrEmpty. Replace all calls to GetNamespace with NamespaceOrEmpty or NamespaceOrDefault as appropriate.	4 years ago
Daniel Nephin	1d2d15b1e1	agent: add a test for streaming in the service health endpoint ... Co-authored-by: Paul Banks <banks@banksco.de>	4 years ago
Daniel Nephin	d1772ae305	structs: rename EnterpriseMeta constructor ... To match the Go convention.	4 years ago
Daniel Nephin	b9e60c0775	testing: skip slow tests with -short ... Add a skip condition to all tests slower than 100ms. This change was made using `gotestsum tool slowest` with data from the last 3 CI runs of master. See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests With this change: ``` $ time go test -count=1 -short ./agent ok github.com/hashicorp/consul/agent 0.743s real 0m4.791s $ time go test -count=1 -short ./agent/consul ok github.com/hashicorp/consul/agent/consul 4.229s real 0m8.769s ```	4 years ago
Daniel Nephin	fb70c8bac2	stream: document that Payload must be immutable ... If they are sent to EventPublisher.Publish. Also document that PayloadEvents is expected to come from a subscription and that it is not immutable.	4 years ago
Daniel Nephin	868cfe1eac	stream: Add HasReadPermission to Payload ... Required now that filter is a method on PayloadEvents instead of Event	4 years ago
Daniel Nephin	a33c50ef0d	Merge pull request #9073 from hashicorp/dnephin/backport-streaming-namespaces ... streaming: backport namespace changes	4 years ago
Daniel Nephin	c82f6ef2d8	Merge pull request #9061 from hashicorp/dnephin/event-fields ... stream: support filtering by namespace	4 years ago
Daniel Nephin	b532e092dc	structs: add a namespace test for CheckServiceNode.CanRead	4 years ago
Daniel Nephin	c42fe5ae43	subscribe: set the request namespace	4 years ago
Daniel Nephin	a5dd2001cf	stream: remove Event.Key ... Makes Payload a type with FilterByKey so that Payloads can implement filtering by key. With this approach we don't need to expose a Namespace field on Event, and we don't need to invest micro formats or require a bunch of code to be aware of exactly how the key field is encoded.	4 years ago
Daniel Nephin	68342a0cb5	proto: remove Event.Key field ... The field is never used, and the value is available from the payload.	4 years ago
Daniel Nephin	9a1e845be8	proto: remove Event.Namespace field ... All events are part of a single Topic, so we don't need this field.	4 years ago
Daniel Nephin	3dfb7c224b	stream: Use a no-op event publisher if streaming is disabled	4 years ago
Daniel Nephin	fb57d9b26a	stream: close the subscription on Unsubscribe	4 years ago
Daniel Nephin	c5d57c9f07	subscribe: add test cases for newEventFromStreamEvent	4 years ago
Daniel Nephin	f185124320	subscribe: Add steps to rpc/subscribe tests ... To make them easier to follow	4 years ago
Daniel Nephin	b103568e98	Merge pull request #8818 from hashicorp/streaming/add-subscribe-service-batch-events ... stream: handle batch events as a special case of Event	4 years ago
Daniel Nephin	21c21191f4	structs: add CheckServiceNode.CanRead ... And use it from the subscribe endpoint.	4 years ago
Daniel Nephin	b27068b72a	stream: Return a single event from a subscription.Next ... Handle batch events as a single event	4 years ago
Daniel Nephin	f5d11562f2	subscribe: update to use NewSnapshotToFollow event	4 years ago
Daniel Nephin	e3290f5971	Move agent/subscribe -> agent/rpc/subscribe	4 years ago