consul

Commit Graph

Author	SHA1	Message	Date
Semir Patel	83a2e726dc	[BACKPORT] 1.15.x manual backport of OSS->CE branch (#18552 ) OSS -> CE (community edition) changes (#18517)	1 year ago
hc-github-team-consul-core	a5960414a1	Backport of [OSS] gRPC Blocking Queries into release/1.15.x (#17441 ) * backport of feat: initial grpc blocking queries (`11b1dd2`) * backport of changelog and docs update (`5ceef20`) --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>	2 years ago
hc-github-team-consul-core	89ce3ba123	backport of commit `78bb205fc3` (#16635 ) Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>	2 years ago
Derek Menteer	19a46d6ca4	Enforce lowercase peer names. (#15697 ) Enforce lowercase peer names. Prior to this change peer names could be mixed case. This can cause issues, as peer names are used as DNS labels in various locations. It also caused issues with envoy configuration.	2 years ago
Matt Keeler	5afd4657ec	Protobuf Modernization (#15949 ) * Protobuf Modernization Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf Marshallers (protobuf and json) needed some changes to account for different APIs. Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package. This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces. * Fix go-mod-tidy make target to work on all modules	2 years ago
Dan Stough	626249fbf5	[OSS] fix: wait and try longer to peer through mesh gw (#15328 )	2 years ago
Derek Menteer	b64972d486	Bring back parameter ServerExternalAddresses in GenerateToken endpoint (#15267 ) Re-add ServerExternalAddresses parameter in GenerateToken endpoint This reverts commit `5e156772f6` and adds extra functionality to support newer peering behaviors.	2 years ago
malizz	617a5f2dc2	convert stream status time fields to pointers (#15252 )	2 years ago
freddygv	d65e60de86	Return forbidden on permission denied This commit updates the establish endpoint to bubble up a 403 status code to callers when the establishment secret from the token is invalid. This is a signal that a new peering token must be generated.	2 years ago
Nitya Dhanushkodi	5e156772f6	Remove ability to specify external addresses in GenerateToken endpoint (#14930 ) * Reverts "update generate token endpoint to take external addresses (#13844)" This reverts commit `f47319b7c6`.	2 years ago
freddygv	e69bc727ec	Update peering establishment to maybe use gateways When peering through mesh gateways we expect outbound dials to peer servers to flow through the local mesh gateway addresses. Now when establishing a peering we get a list of dial addresses as a ring buffer that includes local mesh gateway addresses if the local DC is configured to peer through mesh gateways. The ring buffer includes the mesh gateway addresses first, but also includes the remote server addresses as a fallback. This fallback is present because it's possible that direct egress from the servers may be allowed. If not allowed then the leader will cycle back to a mesh gateway address through the ring. When attempting to dial the remote servers we retry up to a fixed timeout. If using mesh gateways we also have an initial wait in order to allow for the mesh gateways to configure themselves. Note that if we encounter a permission denied error we do not retry since that error indicates that the secret in the peering token is invalid.	2 years ago
Derek Menteer	4e140c98bc	Address PR comments.	2 years ago
Derek Menteer	1e394da400	Disallow peering to the same cluster.	2 years ago
Derek Menteer	caa1396255	Add remote peer partition and datacenter info.	2 years ago
Chris S. Kim	4f4112662e	Fix nil pointer	2 years ago
Chris S. Kim	b0a4c5c563	Include stream-related information in peering endpoints	2 years ago
Freddy	4abad02abd	Merge pull request #14796 from hashicorp/peering/use-connect-ca	2 years ago
freddygv	3034df6a5c	Require Connect and TLS to generate peering tokens By requiring Connect and a gRPC TLS listener we can automatically configure TLS for all peering control-plane traffic.	2 years ago
freddygv	fac3ddc857	Use internal server certificate for peering TLS A previous commit introduced an internally-managed server certificate to use for peering-related purposes. Now the peering token has been updated to match that behavior: - The server name matches the structure of the server cert - The CA PEMs correspond to the Connect CA Note that if Conect is disabled, and by extension the Connect CA, we fall back to the previous behavior of returning the manually configured certs and local server SNI. Several tests were updated to use the gRPC TLS port since they enable Connect by default. This means that the peering token will embed the Connect CA, and the dialer will expect a TLS listener.	2 years ago
DanStough	77ab28c5c7	feat: xDS updates for peerings control plane through mesh gw	2 years ago
freddygv	4ff9d475b0	Return mesh gateway addrs if peering through mgw	2 years ago
Eric Haberkorn	80e51ff907	Add exported services event to cluster peering replication. (#14797 )	2 years ago
malizz	84b0f408fa	Support Stale Queries for Trust Bundle Lookups (#14724 ) * initial commit * add tags, add conversations * add test for query options utility functions * update previous tests * fix test * don't error out on empty context * add changelog * update decode config	2 years ago
DanStough	485e1b5d4e	fix(peering): generate token metrics only for leader	2 years ago
DanStough	2a2debee64	feat(peering): validate server name conflicts on establish	2 years ago
freddygv	650e48624d	Allow terminated peerings to be deleted Peerings are terminated when a peer decides to delete the peering from their end. Deleting a peering sends a termination message to the peer and triggers them to mark the peering as terminated but does NOT delete the peering itself. This is to prevent peerings from disappearing from both sides just because one side deleted them. Previously the Delete endpoint was skipping the deletion if the peering was not marked as active. However, terminated peerings are also inactive. This PR makes some updates so that peerings marked as terminated can be deleted by users.	2 years ago
Chris S. Kim	6ddcc04613	Replace ring buffer with async version (#14314 ) We need to watch for changes to peerings and update the server addresses which get served by the ring buffer. Also, if there is an active connection for a peer, we are getting up-to-date server addresses from the replication stream and can safely ignore the token's addresses which may be stale.	2 years ago
freddygv	c04515a844	Use proto message for each secrets write op Previously there was a field indicating the operation that triggered a secrets write. Now there is a message for each operation and it contains the secret ID being persisted.	2 years ago
freddygv	60d6e28c97	Pass explicit signal with op for secrets write Previously the updates to the peering secrets UUID table relied on inferring what action triggered the update based on a reconciliation against the existing secrets. Instead we now explicitly require the operation to be given so that the inference isn't necessary. This makes the UUID table logic easier to reason about and fixes some related bugs. There is also an update so that the peering secrets get handled on snapshots/restores.	2 years ago
freddygv	9ca687bc7c	Avoid deleting peering secret UUIDs at dialers Dialers do not keep track of peering secret UUIDs, so they should not attempt to clean up data from that table when their peering is deleted. We also now keep peer server addresses when marking peerings for deletion. Peer server addresses are used by the ShouldDial() helper when determining whether the peering is for a dialer or an acceptor. We need to keep this data so that peering secrets can be cleaned up accordingly.	2 years ago
Matt Keeler	f74d0cef7a	Implement/Utilize secrets for Peering Replication Stream (#13977 )	2 years ago
acpana	6033584349	use EqualPartitions Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
acpana	0351ca5136	better fix Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
acpana	8b2ef80336	sync w ent Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
alex	437a28d18a	peering: prevent peering in same partition (#13851 ) Co-authored-by: Chris S. Kim <ckim@hashicorp.com>	2 years ago
Luke Kysow	3530d3782d	peering: read endpoints can now return failing status (#13849 ) Track streams that have been disconnected due to an error and set their statuses to failing.	2 years ago
Chris S. Kim	73a84f256f	Preserve PeeringState on upsert (#13666 ) Fixes a bug where if the generate token is called twice, the second call upserts the zero-value (undefined) of PeeringState.	2 years ago
freddygv	b544ce6485	Add ACL enforcement to peering endpoints	2 years ago
alex	279d458e6e	peering: use ShouldDial to validate peer role (#13823 ) Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
Luke Kysow	a1e6d69454	peering: add config to enable/disable peering (#13867 ) * peering: add config to enable/disable peering Add config: ``` peering { enabled = true } ``` Defaults to true. When disabled: 1. All peering RPC endpoints will return an error 2. Leader won't start its peering establishment goroutines 3. Leader won't start its peering deletion goroutines	2 years ago
Nitya Dhanushkodi	f47319b7c6	update generate token endpoint to take external addresses (#13844 ) Update generate token endpoint (rpc, http, and api module) If ServerExternalAddresses are set, it will override any addresses gotten from the "consul" service, and be used in the token instead, and dialed by the dialer. This allows for setting up a load balancer for example, in front of the consul servers.	2 years ago
alex	de5a991d8c	peering: refactor reconcile, cleanup (#13795 ) Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
alex	a9ae2ff4fa	peering: track exported services (#13784 ) Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
alex	adb5ffa1a6	peering: track imported services (#13718 )	2 years ago
Dan Upton	b9e525d689	grpc: rename public/private directories to external/internal (#13721 ) Previously, public referred to gRPC services that are both exposed on the dedicated gRPC port and have their definitions in the proto-public directory (so were considered usable by 3rd parties). Whereas private referred to services on the multiplexed server port that are only usable by agents and other servers. Now, we're splitting these definitions, such that external/internal refers to the port and public/private refers to whether they can be used by 3rd parties. This is necessary because the peering replication API needs to be exposed on the dedicated port, but is not (yet) suitable for use by 3rd parties.	2 years ago
R.B. Boyer	af04851637	peering: move peer replication to the external gRPC port (#13698 ) Peer replication is intended to be between separate Consul installs and effectively should be considered "external". This PR moves the peer stream replication bidirectional RPC endpoint to the external gRPC server and ensures that things continue to function.	2 years ago
Chris S. Kim	f07132dacc	Revise possible states for a peering. (#13661 ) These changes are primarily for Consul's UI, where we want to be more specific about the state a peering is in. - The "initial" state was renamed to pending, and no longer applies to peerings being established from a peering token. - Upon request to establish a peering from a peering token, peerings will be set as "establishing". This will help distinguish between the two roles: the cluster that generates the peering token and the cluster that establishes the peering. - When marked for deletion, peering state will be set to "deleting". This way the UI determines the deletion via the state rather than the "DeletedAt" field. Co-authored-by: freddygv <freddy@hashicorp.com>	2 years ago
alex	beb8b03e8a	peering: reconcile/ hint active state for list (#13619 ) Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
R.B. Boyer	0fa828db76	peering: replicate all SpiffeID values necessary for the importing side to do SAN validation (#13612 ) When traversing an exported peered service, the discovery chain evaluation at the other side may re-route the request to a variety of endpoints. Furthermore we intend to terminate mTLS at the mesh gateway for arriving peered traffic that is http-like (L7), so the caller needs to know the mesh gateway's SpiffeID in that case as well. The following new SpiffeID values will be shipped back in the peerstream replication: - tcp: all possible SpiffeIDs resulting from the service-resolver component of the exported discovery chain - http-like: the SpiffeID of the mesh gateway	2 years ago
R.B. Boyer	e8ea3d7c3b	state: peering ID assignment cannot happen inside of the state store (#13525 ) Move peering ID assignment outisde of the FSM, so that the ID is written to the raft log and the same ID is used by all voters, and after restarts.	2 years ago

1 2

73 Commits (d1bc9fbbf3d298f0f4172432dc73709c0c1e9364)