216 Commits (jm/retract)

Author	SHA1	Message	Date
freddygv	7f9a5d0f58	Add basic nonce management ... This commit adds a monotonically increasing nonce to include in peering replication response messages. Every ack/nack from the peer handling a response will include this nonce, allowing to correlate the ack/nack with a specific resource. At the moment nothing is done with the nonce when it is received. In the future we may want to add functionality such as retries on NACKs, depending on the class of error.	2 years ago
Paul Glass	d17af23641	gRPC server metrics (#14922) ... * Move stats.go from grpc-internal to grpc-middleware * Update grpc server metrics with server type label * Add stats test to grpc-external * Remove global metrics instance from grpc server tests	2 years ago
freddygv	bf72df7b0e	Fixup test	2 years ago
Chris S. Kim	b0a4c5c563	Include stream-related information in peering endpoints	2 years ago
Paul Glass	c0c187f1c5	Merge central config for GetEnvoyBootstrapParams (#14869) ... This fixes GetEnvoyBootstrapParams to merge in proxy-defaults and service-defaults. Co-authored-by: Dan Upton <daniel@floppy.co>	2 years ago
freddygv	3034df6a5c	Require Connect and TLS to generate peering tokens ... By requiring Connect and a gRPC TLS listener we can automatically configure TLS for all peering control-plane traffic.	2 years ago
Eric Haberkorn	1b565444be	Rename `PeerName` to `Peer` on prepared queries and exported services (#14854)	2 years ago
freddygv	a8c4d6bc55	Share mgw addrs in peering stream if needed ... This commit adds handling so that the replication stream considers whether the user intends to peer through mesh gateways. The subscription will return server or mesh gateway addresses depending on the mesh configuration setting. These watches can be updated at runtime by modifying the mesh config entry.	2 years ago
Chris Chapman	d7b5351b66	Making suggested comments	2 years ago
Chris Chapman	46bea72212	Making suggested changes	2 years ago
Chris Chapman	a05563b788	Update comment	2 years ago
Chris Chapman	7bc9cad180	Adding grpc handler for dns proxy	2 years ago
Eric Haberkorn	80e51ff907	Add exported services event to cluster peering replication. (#14797)	2 years ago
malizz	84b0f408fa	Support Stale Queries for Trust Bundle Lookups (#14724) ... * initial commit * add tags, add conversations * add test for query options utility functions * update previous tests * fix test * don't error out on empty context * add changelog * update decode config	2 years ago
Dan Upton	1c2c975b0b	xDS Load Balancing (#14397) ... Prior to #13244, connect proxies and gateways could only be configured by an xDS session served by the local client agent. In an upcoming release, it will be possible to deploy a Consul service mesh without client agents. In this model, xDS sessions will be handled by the servers themselves, which necessitates load-balancing to prevent a single server from receiving a disproportionate amount of load and becoming overwhelmed. This introduces a simple form of load-balancing where Consul will attempt to achieve an even spread of load (xDS sessions) between all healthy servers. It does so by implementing a concurrent session limiter (limiter.SessionLimiter) and adjusting the limit according to autopilot state and proxy service registrations in the catalog. If a server is already over capacity (i.e. the session limit is lowered), Consul will begin draining sessions to rebalance the load. This will result in the client receiving a `RESOURCE_EXHAUSTED` status code. It is the client's responsibility to observe this response and reconnect to a different server. Users of the gRPC client connection brokered by the consul-server-connection-manager library will get this for free. The rate at which Consul will drain sessions to rebalance load is scaled dynamically based on the number of proxies in the catalog.	2 years ago
Derek Menteer	f7c884f0af	Merge branch 'main' of github.com:hashicorp/consul into derekm/split-grpc-ports	2 years ago
Derek Menteer	bf769daae4	Merge branch 'main' of github.com:hashicorp/consul into derekm/split-grpc-ports	2 years ago
Chris S. Kim	953808e899	PR feedback on terminated state checking	2 years ago
Chris S. Kim	ec36755cc0	Properly assert for ServerAddresses replication request	2 years ago
Chris S. Kim	d1d9dbff8e	Fix terminate not returning early	2 years ago
Derek Menteer	f64771c707	Address PR comments.	2 years ago
Chris S. Kim	560d410c6d	Merge branch 'main' into NET-638-push-server-address-updates-to-the-peer ... # Conflicts: # agent/grpc-external/services/peerstream/stream_test.go	2 years ago
Chris S. Kim	74ddf040dd	Add heartbeat timeout grace period when accounting for peering health	2 years ago
Derek Menteer	1255a8a20d	Add separate grpc_tls port. ... To ease the transition for users, the original gRPC port can still operate in a deprecated mode as either plain-text or TLS mode. This behavior should be removed in a future release whenever we no longer support this. The resulting behavior from this commit is: `ports.grpc > 0 && ports.grpc_tls > 0` spawns both plain-text and tls ports. `ports.grpc > 0 && grpc.tls == undefined` spawns a single plain-text port. `ports.grpc > 0 && grpc.tls != undefined` spawns a single tls port (backwards compat mode).	2 years ago
Chris S. Kim	def529edd3	Rename test	2 years ago
Chris S. Kim	93271f649c	Fix test	2 years ago
Chris S. Kim	4d97e2f936	Adjust metrics reporting for peering tracker	2 years ago
Chris S. Kim	937a8ec742	Fix casing	2 years ago
Chris S. Kim	1c43a1a7b4	Merge branch 'main' into NET-638-push-server-address-updates-to-the-peer ... # Conflicts: # agent/grpc-external/services/peerstream/stream_test.go	2 years ago
alex	30ff2e9a35	peering: add peer health metric (#14004) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
Chris S. Kim	8c94d1a80c	Update test comment	2 years ago
Chris S. Kim	5f2959329f	Add check for zero-length server addresses	2 years ago
Pablo Ruiz García	1f293e5244	Added new auto_encrypt.grpc_server_tls config option to control AutoTLS enabling of GRPC Server's TLS usage ... Fix for #14253 Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>	2 years ago
Dan Upton	3b993f2da7	dataplane: update envoy bootstrap params for consul-dataplane (#14017) ... Contains 2 changes to the GetEnvoyBootstrapParams response to support consul-dataplane. Exposing node_name and node_id: consul-dataplane will support providing either the node_id or node_name in its configuration. Unfortunately, supporting both in the xDS meta adds a fair amount of complexity (partly because most tables are currently indexed on node_name) so for now we're going to return them both from the bootstrap params endpoint, allowing consul-dataplane to exchange a node_id for a node_name (which it will supply in the xDS meta). Properly setting service for gateways: To avoid the need to special case gateways in consul-dataplane, service will now either be the destination service name for connect proxies, or the gateway service name. This means it can be used as-is in Envoy configuration (i.e. as a cluster name or in metric tags).	2 years ago
Chris S. Kim	cdc8b0634d	Fix flakes	2 years ago
Chris S. Kim	03e92826aa	Increase heartbeat rate to reduce test flakes	2 years ago
Chris S. Kim	06ba9775ee	Remove check for ResponseNonce	2 years ago
Chris S. Kim	adff2eef16	Fix data race ... newMockSnapshotHandler has an assertion on t.Cleanup which gets called before the event publisher is cancelled. This commit reorders the context.WithCancel so it properly gets cancelled before the assertion is made.	2 years ago
Chris S. Kim	4e40e1d222	Handle server addresses update as client	2 years ago
Chris S. Kim	584d3409c4	Send server addresses on update from server	2 years ago
Chris S. Kim	c9d8ad3939	Add new subscription for server addresses	2 years ago
Chris S. Kim	028b87d51f	Cleanup unused logger	2 years ago
freddygv	1031ffc3c7	Re-validate existing secrets at state store ... Previously establishment and pending secrets were only checked at the RPC layer. However, given that these are Check-and-Set transactions we should ensure that the given secrets are still valid when persisting a secret exchange or promotion. Otherwise it would be possible for concurrent requests to overwrite each other.	2 years ago
freddygv	c04515a844	Use proto message for each secrets write op ... Previously there was a field indicating the operation that triggered a secrets write. Now there is a message for each operation and it contains the secret ID being persisted.	2 years ago
freddygv	60d6e28c97	Pass explicit signal with op for secrets write ... Previously the updates to the peering secrets UUID table relied on inferring what action triggered the update based on a reconciliation against the existing secrets. Instead we now explicitly require the operation to be given so that the inference isn't necessary. This makes the UUID table logic easier to reason about and fixes some related bugs. There is also an update so that the peering secrets get handled on snapshots/restores.	2 years ago
Freddy	42996411cc	Various peering fixes (#13979) ... * Avoid logging StreamSecretID * Wrap additional errors in stream handler * Fix flakiness in leader test and rename servers for clarity. There was a race condition where the peering was being deleted in the test before the stream was active. Now the test waits for the stream to be connected on both sides before deleting the associated peering. * Run flaky test serially	2 years ago
Matt Keeler	f74d0cef7a	Implement/Utilize secrets for Peering Replication Stream (#13977)	2 years ago
Luke Kysow	95096e2c03	peering: retry establishing connection more quickly on certain errors (#13938) ... When we receive a FailedPrecondition error, retry that more quickly because we expect it will resolve shortly. This is particularly important in the context of Consul servers behind a load balancer because when establishing a connection we have to retry until we randomly land on a leader node. The default retry backoff goes from 2s, 4s, 8s, etc. which can result in very long delays quite quickly. Instead, this backoff retries in 8ms five times, then goes exponentially from there: 16ms, 32ms, ... up to a max of 8152ms.	2 years ago
Luke Kysow	465a9801e1	Merge pull request #13924 from hashicorp/lkysow/util-metric-peering ... peering: don't track imported services/nodes in usage	2 years ago
Luke Kysow	021b00e321	Remove duplicate comment	2 years ago
Luke Kysow	8c5b70d227	Rename receive to recv in tracker (#13896) ... Because it's shorter	2 years ago
Luke Kysow	3530d3782d	peering: read endpoints can now return failing status (#13849) ... Track streams that have been disconnected due to an error and set their statuses to failing.	2 years ago
alex	927cee692b	peering: emit exported services count metric (#13811) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
Luke Kysow	0c87be0845	peering: Add heartbeating to peering streams (#13806) ... * Add heartbeating to peering streams	2 years ago
Luke Kysow	c411e6b326	Add send mutex to protect against concurrent sends (#13805)	2 years ago
alex	de5a991d8c	peering: refactor reconcile, cleanup (#13795) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
Luke Kysow	e8d965e56f	peerstream: set keepalive enforcement to 15s (#13796) ... The client is set to send keepalive pings every 30s. The server keepalive enforcement must be set to a number less than that, otherwise it will disconnect clients for sending pings too often. MinTime governs the minimum amount of time between pings.	2 years ago
alex	a9ae2ff4fa	peering: track exported services (#13784) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
R.B. Boyer	cd513aeead	peerstream: require a resource subscription to receive updates of that type (#13767) ... This mimics xDS's discovery protocol where you must request a resource explicitly for the exporting side to send those events to you. As part of this I aligned the overall ResourceURL with the TypeURL that gets embedded into the encoded protobuf Any construct. The CheckServiceNodes is now wrapped in a better named "ExportedService" struct now.	2 years ago
R.B. Boyer	c737301093	peerstream: fix test assertions (#13780)	2 years ago
Luke Kysow	ca3d7c964c	peerstream: dialer should reconnect when stream closes (#13745) ... * peerstream: dialer should reconnect when stream closes If the stream is closed unexpectedly (i.e. when we haven't received a terminated message), the dialer should attempt to re-establish the stream. Previously, the `HandleStream` would return `nil` when the stream was closed. The caller then assumed the stream was terminated on purpose and so didn't reconnect when instead it was stopped unexpectedly and the dialer should have attempted to reconnect.	2 years ago
R.B. Boyer	bb4d4040fb	server: ensure peer replication can successfully use TLS over external gRPC (#13733) ... Ensure that the peer stream replication rpc can successfully be used with TLS activated. Also: - If key material is configured for the gRPC port but HTTPS is not enabled now TLS will still be activated for the gRPC port. - peerstream replication stream opened by the establishing-side will now ignore grpc.WithBlock so that TLS errors will bubble up instead of being awkwardly delayed or suppressed	2 years ago
alex	adb5ffa1a6	peering: track imported services (#13718)	2 years ago
Matt Keeler	257f88d4df	Use Node Name for peering healthSnapshot instead of ID (#13773) ... A Node ID is not a required field with Consul’s data model. Therefore we cannot reliably expect all uses to have it. However the node name is required and must be unique so its equally as good of a key for the internal healthSnapshot node tracking.	2 years ago
Chris S. Kim	b4ffa9ae0c	Scrub VirtualIPs before exporting	2 years ago
Dan Upton	b9e525d689	grpc: rename public/private directories to external/internal (#13721) ... Previously, public referred to gRPC services that are both exposed on the dedicated gRPC port and have their definitions in the proto-public directory (so were considered usable by 3rd parties). Whereas private referred to services on the multiplexed server port that are only usable by agents and other servers. Now, we're splitting these definitions, such that external/internal refers to the port and public/private refers to whether they can be used by 3rd parties. This is necessary because the peering replication API needs to be exposed on the dedicated port, but is not (yet) suitable for use by 3rd parties.	2 years ago