217 Commits (9f949ead34012d6f3e8c776f7efb272e1871ffd3)

Author	SHA1	Message	Date
Semir Patel	1794484298	Resource `Delete` endpoint (#16756)	2 years ago
Dan Upton	4fa2537b3b	Resource `Write` endpoint (#16786)	2 years ago
Dan Upton	671d5825ca	Raft storage backend (#16619)	2 years ago
Dan Upton	651549c97d	storage: fix resource leak in Watch (#16817)	2 years ago
Ronald	b64674623e	Copyright headers for missing files/folders (#16708) ... * copyright headers for agent folder	2 years ago
Ronald	94ec4eb2f4	copyright headers for agent folder (#16704) ... * copyright headers for agent folder * Ignore test data files * fix proto files and remove headers in agent/uiserver folder * ignore deep-copy files	2 years ago
Semir Patel	440f11203f	Resource service List(..) endpoint (#16753)	2 years ago
Semir Patel	032aba3175	WatchList(..) endpoint for the resource service (#16726)	2 years ago
Semir Patel	3415689eb6	Read(...) endpoint for the resource service (#16655)	2 years ago
Semir Patel	176945aa86	GRPC stub for the ResourceService (#16528)	2 years ago
Chris S. Kim	a518893685	Fix various flaky tests (#16396)	2 years ago
Derek Menteer	ad865f549b	Fix issue with peer services incorrectly appearing as connect-enabled. (#16339) ... Prior to this commit, all peer services were transmitted as connect-enabled as long as a one or more mesh-gateways were healthy. With this change, there is now a difference between typical services and connect services transmitted via peering. A service will be reported as "connect-enabled" as long as any of these conditions are met: 1. a connect-proxy sidecar is registered for the service name. 2. a connect-native instance of the service is registered. 3. a service resolver / splitter / router is registered for the service name. 4. a terminating gateway has registered the service.	2 years ago
Matt Keeler	085c0addc0	Protobuf Refactoring for Multi-Module Cleanliness (#16302) ... Protobuf Refactoring for Multi-Module Cleanliness This commit includes the following: Moves all packages that were within proto/ to proto/private Rewrites imports to account for the packages being moved Adds in buf.work.yaml to enable buf workspaces Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes) Why: In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage. There were some recent changes to have our own ratelimiting annotations. The two combined were not working when I was trying to use them together (attempting to rebase another branch) Buf workspaces should be the solution to the problem Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root. This resulted in proto file name conflicts in the Go global protobuf type registry. The solution to that was to add in a private/ directory into the path within the proto/ directory. That then required rewriting all the imports. Is this safe? AFAICT yes The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc) Other than imports, there were no changes to any generated code as a result of this.	2 years ago
Dan Stough	f1436109ea	[OSS] security: update go to 1.20.1 (#16263) ... * security: update go to 1.20.1	2 years ago
Nathan Coleman	72a73661c9	Implement APIGateway proxycfg snapshot (#16194) ... * Stub proxycfg handler for API gateway * Add Service Kind constants/handling for API Gateway * Begin stubbing for SDS * Add new Secret type to xDS order of operations * Continue stubbing of SDS * Iterate on proxycfg handler for API gateway * Handle BoundAPIGateway config entry subscription in proxycfg-glue * Add API gateway to config snapshot validation * Add API gateway to config snapshot clone, leaf, etc. * Subscribe to bound route + cert config entries on bound-api-gateway * Track routes + certs on API gateway config snapshot * Generate DeepCopy() for types used in watch.Map * Watch all active references on api-gateway, unwatch inactive * Track loading of initial bound-api-gateway config entry * Use proper proto package for SDS mapping * Use ResourceReference instead of ServiceName, collect resources * Fix typo, add + remove TODOs * Watch discovery chains for TCPRoute * Add TODO for updating gateway services for api-gateway * make proto * Regenerate deep-copy for proxycfg * Set datacenter on upstream ID from query source * Watch discovery chains for http-route service backends * Add ServiceName getter to HTTP+TCP Service structs * Clean up unwatched discovery chains on API Gateway * Implement watch for ingress leaf certificate * Collect upstreams on http-route + tcp-route updates * Remove unused GatewayServices update handler * Remove unnecessary gateway services logic for API Gateway * Remove outdate TODO * Use .ToIngress where appropriate, including TODO for cleaning up * Cancel before returning error * Remove GatewayServices subscription * Add godoc for handlerAPIGateway functions * Update terminology from Connect => Consul Service Mesh Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690 * Add missing TODO * Remove duplicate switch case * Rerun deep-copy generator * Use correct property on config snapshot * Remove unnecessary leaf cert watch * Clean up based on code review feedback * Note handler properties that are initialized but set elsewhere * Add TODO for moving helper func into structs pkg * Update generated DeepCopy code * gofmt * Generate DeepCopy() for API gateway listener types * Improve variable name * Regenerate DeepCopy() code * Fix linting issue * Temporarily remove the secret type from resource generation	2 years ago
skpratt	a010902978	Remove legacy acl policies (#15922) ... * remove legacy tokens * remove legacy acl policies * flatten test policies to *_prefix * address oss feedback re: phrasing and tests	2 years ago
Dan Upton	7a55de375c	xds: don't attempt to load-balance sessions for local proxies (#15789) ... Previously, we'd begin a session with the xDS concurrency limiter regardless of whether the proxy was registered in the catalog or in the server's local agent state. This caused problems for users who run `consul connect envoy` directly against a server rather than a client agent, as the server's locally registered proxies wouldn't be included in the limiter's capacity. Now, the `ConfigSource` is responsible for beginning the session and we only do so for services in the catalog. Fixes: https://github.com/hashicorp/consul/issues/15753	2 years ago
Chris S. Kim	e4a268e33e	Warn if ACL is enabled but no token is provided to Envoy (#15967)	2 years ago
Dan Stough	6d2880e894	feat: add access logs to dataplane bootstrap rpc (#15951)	2 years ago
Matt Keeler	5afd4657ec	Protobuf Modernization (#15949) ... * Protobuf Modernization Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf Marshallers (protobuf and json) needed some changes to account for different APIs. Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package. This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces. * Fix go-mod-tidy make target to work on all modules	2 years ago
Chris S. Kim	a7b34d50fc	Output user-friendly name for anonymous token (#15884)	2 years ago
Dan Upton	b78de5a7a2	grpc/acl: fix bug where ACL token was required even if disabled (#15904) ... Fixes a bug introduced by #15346 where we'd always require an ACL token even if ACLs were disabled because we were erroneously treating `nil` identity as anonymous.	2 years ago
Dan Upton	d4c435856b	grpc: `protoc` plugin for generating gRPC rate limit specifications (#15564) ... Adds automation for generating the map of `gRPC Method Name → Rate Limit Type` used by the middleware introduced in #15550, and will ensure we don't forget to add new endpoints. Engineers must annotate their RPCs in the proto file like so: ``` rpc Foo(FooRequest) returns (FooResponse) { option (consul.internal.ratelimit.spec) = { operation_type: READ, }; } ``` When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will be installed that writes rate-limit specs as a JSON array to a file called `.ratelimit.tmp` (one per protobuf package/directory). After running Buf, `make proto` will execute a post-process script that will ingest all of the `.ratelimit.tmp` files and generate a Go file containing the mappings in the `agent/grpc-middleware` package. In the enterprise repository, it will write an additional file with the enterprise-only endpoints. If an engineer forgets to add the annotation to a new RPC, the plugin will return an error like so: ``` RPC Foo is missing rate-limit specification, fix it with: import "proto-public/annotations/ratelimit/ratelimit.proto"; service Bar { rpc Foo(...) returns (...) { option (hashicorp.consul.internal.ratelimit.spec) = { operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT, }; } } ``` In the future, this annotation can be extended to support rate-limit category (e.g. KV vs Catalog) and to determine the retry policy.	2 years ago
Dan Upton	7c7503c849	grpc/acl: relax permissions required for "core" endpoints (#15346) ... Previously, these endpoints required `service:write` permission on _any_ service as a sort of proxy for "is the caller allowed to participate in the mesh?". Now, they're called as part of the process of establishing a server connection by any consumer of the consul-server-connection-manager library, which will include non-mesh workloads (e.g. Consul KV as a storage backend for Vault) as well as ancillary components such as consul-k8s' acl-init process, which likely won't have `service:write` permission. So this commit relaxes those requirements to accept *any* valid ACL token on the following gRPC endpoints: - `hashicorp.consul.dataplane.DataplaneService/GetSupportedDataplaneFeatures` - `hashicorp.consul.serverdiscovery.ServerDiscoveryService/WatchServers` - `hashicorp.consul.connectca.ConnectCAService/WatchRoots`	2 years ago
John Murret	f5e01f8c6b	Rate Limit Handler - ensure rate limiting is not in the code path when not configured (#15819) ... * Rate limiting handler - ensure configuration has changed before modifying limiters * Updating test to validate arguments to UpdateConfig * Removing duplicate test. Updating mock. * Renaming NullRateLimiter to NullRequestLimitsHandler * Rate Limit Handler - ensure rate limiting is not in the code path when not configured * Update agent/consul/rate/handler.go Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * formatting handler.go * Rate limiting handler - ensure configuration has changed before modifying limiters * Updating test to validate arguments to UpdateConfig * Removing duplicate test. Updating mock. * adding logging for when UpdateConfig is called but the config has not changed. * Update agent/consul/rate/handler.go Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * Update agent/consul/rate/handler_test.go Co-authored-by: Dan Upton <daniel@floppy.co> * modifying existing variable name based on pr feedback * updating a broken merge conflict; Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> Co-authored-by: Dan Upton <daniel@floppy.co>	2 years ago
John Murret	e027c94b52	adding config for request_limits (#15531) ... * server: add placeholder glue for rate limit handler This commit adds a no-op implementation of the rate-limit handler and adds it to the `consul.Server` struct and setup code. This allows us to start working on the net/rpc and gRPC interceptors and config logic. * Add handler errors * Set the global read and write limits * fixing multilimiter moving packages * Fix typo * Simplify globalLimit usage * add multilimiter and tests * exporting LimitedEntity * Apply suggestions from code review Co-authored-by: John Murret <john.murret@hashicorp.com> * add config update and rename config params * add doc string and split config * Apply suggestions from code review Co-authored-by: Dan Upton <daniel@floppy.co> * use timer to avoid go routine leak and change the interface * add comments to tests * fix failing test * add prefix with config edge, refactor tests * Apply suggestions from code review Co-authored-by: Dan Upton <daniel@floppy.co> * refactor to apply configs for limiters under a prefix * add fuzz tests and fix bugs found. Refactor reconcile loop to have a simpler logic * make KeyType an exported type * split the config and limiter trees to fix race conditions in config update * rename variables * fix race in test and remove dead code * fix reconcile loop to not create a timer on each loop * add extra benchmark tests and fix tests * fix benchmark test to pass value to func * server: add placeholder glue for rate limit handler This commit adds a no-op implementation of the rate-limit handler and adds it to the `consul.Server` struct and setup code. This allows us to start working on the net/rpc and gRPC interceptors and config logic. * Set the global read and write limits * fixing multilimiter moving packages * add server configuration for global rate limiting. * remove agent test * remove added stuff from handler * remove added stuff from multilimiter * removing unnecessary TODOs * Removing TODO comment from handler * adding in defaulting to infinite * add disabled status in there * adding in documentation for disabled mode. * make disabled the default. * Add mock and agent test * addig documentation and missing mock file. * Fixing test TestLoad_IntegrationWithFlags * updating docs based on PR feedback. * Updating Request Limits mode to use int based on PR feedback. * Adding RequestLimits struct so we have a nested struct in ReloadableConfig. * fixing linting references * Update agent/consul/rate/handler.go Co-authored-by: Dan Upton <daniel@floppy.co> * Update agent/consul/config.go Co-authored-by: Dan Upton <daniel@floppy.co> * removing the ignore of the request limits in JSON. addingbuilder logic to convert any read rate or write rate less than 0 to rate.Inf * added conversion function to convert request limits object to handler config. * Updating docs to reflect gRPC and RPC are rate limit and as a result, HTTP requests are as well. * Updating values for TestLoad_FullConfig() so that they were different and discernable. * Updating TestRuntimeConfig_Sanitize * Fixing TestLoad_IntegrationWithFlags test * putting nil check in place * fixing rebase * removing change for missing error checks. will put in another PR * Rebasing after default multilimiter config change * resolving rebase issues * updating reference for incomingRPCLimiter to use interface * updating interface * Updating interfaces * Fixing mock reference Co-authored-by: Daniel Upton <daniel@floppy.co> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
Dan Upton	c692802dec	grpc: add rate-limiting middleware (#15550) ... Implements the gRPC middleware for rate-limiting as a tap.ServerInHandle function (executed before the request is unmarshaled). Mappings between gRPC methods and their operation type are generated by a protoc plugin introduced by #15564.	2 years ago
R.B. Boyer	11a277f372	peering: better represent non-passing states during peer check flattening (#15615) ... During peer stream replication we flatten checks from the source cluster and build one thin overall check to hide the irrelevant details from the consuming cluster. This flattening logic did correctly flip to non-passing if there were any non-passing checks, but WHICH status it got during that was random (warn/error). Also it didn't represent "maintenance" operations. There is an api package call AggregatedStatus which more correctly flattened check statuses. This PR replicated the more complete logic into the peer stream package.	2 years ago
Dan Upton	7b2d08d461	chore: remove unused argument from MergeNodeServiceWithCentralConfig (#15024) ... Previously, the MergeNodeServiceWithCentralConfig method accepted a ServiceSpecificRequest argument, of which only the Datacenter and QueryOptions fields were used. Digging a little deeper, it turns out these fields were only passed down to the ComputeResolvedServiceConfig method (through the ServiceConfigRequest struct) which didn't actually use them. As such, not all call-sites passed a valid ServiceSpecificRequest so it's safer to remove the argument altogether to prevent future changes from depending on it.	2 years ago
Derek Menteer	b64972d486	Bring back parameter ServerExternalAddresses in GenerateToken endpoint (#15267) ... Re-add ServerExternalAddresses parameter in GenerateToken endpoint This reverts commit 5e156772f6 and adds extra functionality to support newer peering behaviors.	2 years ago
Chris S. Kim	985a4ee1b1	Update hcp-scada-provider to fix diamond dependency problem with go-msgpack (#15185)	2 years ago
Derek Menteer	18d6c338f4	Backport tests from ent. (#15260) ... * Backport agent tests. Original commit: 0710b2d12fb51a29cedd1119b5fb086e5c71f632 Original commit: aaedb3c28bfe247266f21013d500147d8decb7cd (partial) * Backport test fix and reduce flaky failures.	2 years ago
Derek Menteer	f4cb2f82bf	Backport various fixes from ENT. (#15254) ... * Regenerate golden files. * Backport from ENT: "Avoid race" Original commit: 5006c8c858b0e332be95271ef9ba35122453315b Original author: freddygv * Backport from ENT: "chore: fix flake peerstream test" Original commit: b74097e7135eca48cc289798c5739f9ef72c0cc8 Original author: DanStough	2 years ago
malizz	617a5f2dc2	convert stream status time fields to pointers (#15252)	2 years ago
Derek Menteer	bd1019fadb	Prevent peering acceptor from subscribing to addr updates. (#15214)	2 years ago
Derek Menteer	693c8a4706	Allow peering endpoints to bypass verify_incoming.	2 years ago
cskh	a9427e1310	fix(peering): nil pointer in calling handleUpdateService (#15160) ... * fix(peering): nil pointer in calling handleUpdateService * changelog	2 years ago
Chris S. Kim	bde57c0dd0	Regenerate files according to 1.19.2 formatter	2 years ago
cskh	db82ffe503	fix(peering): replicating wan address (#15108) ... * fix(peering): replicating wan address * add changelog * unit test	2 years ago
R.B. Boyer	300860412c	chore: update golangci-lint to v1.50.1 (#15022)	2 years ago
cskh	d562d363fc	peering: skip registering duplicate node and check from the peer (#14994) ... * peering: skip register duplicate node and check from the peer * Prebuilt the nodes map and checks map to avoid repeated for loop * use key type to struct: node id, service id, and check id	2 years ago
Freddy	24d0c8801a	Merge pull request #14981 from hashicorp/peering/dial-through-gateways	2 years ago
Derek Menteer	0f424e3cdf	Reset wait on ensureServerAddrSubscription	2 years ago
freddygv	96fdd3728a	Fix CA init error code	2 years ago
malizz	b0b0cbb8ee	increase protobuf size limit for cluster peering (#14976)	2 years ago
Derek Menteer	8742fbe14f	Prevent consul peer-exports by discovery chain.	2 years ago
Derek Menteer	f366edcb8d	Prevent the "consul" service from being exported.	2 years ago
Derek Menteer	caa1396255	Add remote peer partition and datacenter info.	2 years ago
Freddy	9ca8bb8ec4	Merge pull request #14958 from hashicorp/peering/nonce	2 years ago
freddygv	1b46b35041	Actually track nonce in test	2 years ago
Derek Menteer	f330438a45	Fix incorrect backoff-wait logic.	2 years ago
freddygv	7f9a5d0f58	Add basic nonce management ... This commit adds a monotonically increasing nonce to include in peering replication response messages. Every ack/nack from the peer handling a response will include this nonce, allowing to correlate the ack/nack with a specific resource. At the moment nothing is done with the nonce when it is received. In the future we may want to add functionality such as retries on NACKs, depending on the class of error.	2 years ago
Paul Glass	d17af23641	gRPC server metrics (#14922) ... * Move stats.go from grpc-internal to grpc-middleware * Update grpc server metrics with server type label * Add stats test to grpc-external * Remove global metrics instance from grpc server tests	2 years ago
freddygv	bf72df7b0e	Fixup test	2 years ago
Chris S. Kim	b0a4c5c563	Include stream-related information in peering endpoints	2 years ago
Paul Glass	c0c187f1c5	Merge central config for GetEnvoyBootstrapParams (#14869) ... This fixes GetEnvoyBootstrapParams to merge in proxy-defaults and service-defaults. Co-authored-by: Dan Upton <daniel@floppy.co>	2 years ago
freddygv	3034df6a5c	Require Connect and TLS to generate peering tokens ... By requiring Connect and a gRPC TLS listener we can automatically configure TLS for all peering control-plane traffic.	2 years ago
Eric Haberkorn	1b565444be	Rename `PeerName` to `Peer` on prepared queries and exported services (#14854)	2 years ago
freddygv	a8c4d6bc55	Share mgw addrs in peering stream if needed ... This commit adds handling so that the replication stream considers whether the user intends to peer through mesh gateways. The subscription will return server or mesh gateway addresses depending on the mesh configuration setting. These watches can be updated at runtime by modifying the mesh config entry.	2 years ago
Chris Chapman	d7b5351b66	Making suggested comments	2 years ago
Chris Chapman	46bea72212	Making suggested changes	2 years ago
Chris Chapman	a05563b788	Update comment	2 years ago
Chris Chapman	7bc9cad180	Adding grpc handler for dns proxy	2 years ago
Eric Haberkorn	80e51ff907	Add exported services event to cluster peering replication. (#14797)	2 years ago
malizz	84b0f408fa	Support Stale Queries for Trust Bundle Lookups (#14724) ... * initial commit * add tags, add conversations * add test for query options utility functions * update previous tests * fix test * don't error out on empty context * add changelog * update decode config	2 years ago
Dan Upton	1c2c975b0b	xDS Load Balancing (#14397) ... Prior to #13244, connect proxies and gateways could only be configured by an xDS session served by the local client agent. In an upcoming release, it will be possible to deploy a Consul service mesh without client agents. In this model, xDS sessions will be handled by the servers themselves, which necessitates load-balancing to prevent a single server from receiving a disproportionate amount of load and becoming overwhelmed. This introduces a simple form of load-balancing where Consul will attempt to achieve an even spread of load (xDS sessions) between all healthy servers. It does so by implementing a concurrent session limiter (limiter.SessionLimiter) and adjusting the limit according to autopilot state and proxy service registrations in the catalog. If a server is already over capacity (i.e. the session limit is lowered), Consul will begin draining sessions to rebalance the load. This will result in the client receiving a `RESOURCE_EXHAUSTED` status code. It is the client's responsibility to observe this response and reconnect to a different server. Users of the gRPC client connection brokered by the consul-server-connection-manager library will get this for free. The rate at which Consul will drain sessions to rebalance load is scaled dynamically based on the number of proxies in the catalog.	2 years ago
Derek Menteer	f7c884f0af	Merge branch 'main' of github.com:hashicorp/consul into derekm/split-grpc-ports	2 years ago
Derek Menteer	bf769daae4	Merge branch 'main' of github.com:hashicorp/consul into derekm/split-grpc-ports	2 years ago
Chris S. Kim	953808e899	PR feedback on terminated state checking	2 years ago
Chris S. Kim	ec36755cc0	Properly assert for ServerAddresses replication request	2 years ago
Chris S. Kim	d1d9dbff8e	Fix terminate not returning early	2 years ago
Derek Menteer	f64771c707	Address PR comments.	2 years ago
Chris S. Kim	560d410c6d	Merge branch 'main' into NET-638-push-server-address-updates-to-the-peer ... # Conflicts: # agent/grpc-external/services/peerstream/stream_test.go	2 years ago
Chris S. Kim	74ddf040dd	Add heartbeat timeout grace period when accounting for peering health	2 years ago
Derek Menteer	1255a8a20d	Add separate grpc_tls port. ... To ease the transition for users, the original gRPC port can still operate in a deprecated mode as either plain-text or TLS mode. This behavior should be removed in a future release whenever we no longer support this. The resulting behavior from this commit is: `ports.grpc > 0 && ports.grpc_tls > 0` spawns both plain-text and tls ports. `ports.grpc > 0 && grpc.tls == undefined` spawns a single plain-text port. `ports.grpc > 0 && grpc.tls != undefined` spawns a single tls port (backwards compat mode).	2 years ago
Chris S. Kim	def529edd3	Rename test	2 years ago
Chris S. Kim	93271f649c	Fix test	2 years ago
Chris S. Kim	4d97e2f936	Adjust metrics reporting for peering tracker	2 years ago
Chris S. Kim	937a8ec742	Fix casing	2 years ago
Chris S. Kim	1c43a1a7b4	Merge branch 'main' into NET-638-push-server-address-updates-to-the-peer ... # Conflicts: # agent/grpc-external/services/peerstream/stream_test.go	2 years ago
alex	30ff2e9a35	peering: add peer health metric (#14004) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
Chris S. Kim	8c94d1a80c	Update test comment	2 years ago
Chris S. Kim	5f2959329f	Add check for zero-length server addresses	2 years ago
Pablo Ruiz García	1f293e5244	Added new auto_encrypt.grpc_server_tls config option to control AutoTLS enabling of GRPC Server's TLS usage ... Fix for #14253 Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>	2 years ago
Dan Upton	3b993f2da7	dataplane: update envoy bootstrap params for consul-dataplane (#14017) ... Contains 2 changes to the GetEnvoyBootstrapParams response to support consul-dataplane. Exposing node_name and node_id: consul-dataplane will support providing either the node_id or node_name in its configuration. Unfortunately, supporting both in the xDS meta adds a fair amount of complexity (partly because most tables are currently indexed on node_name) so for now we're going to return them both from the bootstrap params endpoint, allowing consul-dataplane to exchange a node_id for a node_name (which it will supply in the xDS meta). Properly setting service for gateways: To avoid the need to special case gateways in consul-dataplane, service will now either be the destination service name for connect proxies, or the gateway service name. This means it can be used as-is in Envoy configuration (i.e. as a cluster name or in metric tags).	2 years ago
Chris S. Kim	cdc8b0634d	Fix flakes	2 years ago
Chris S. Kim	03e92826aa	Increase heartbeat rate to reduce test flakes	2 years ago
Chris S. Kim	06ba9775ee	Remove check for ResponseNonce	2 years ago
Chris S. Kim	adff2eef16	Fix data race ... newMockSnapshotHandler has an assertion on t.Cleanup which gets called before the event publisher is cancelled. This commit reorders the context.WithCancel so it properly gets cancelled before the assertion is made.	2 years ago
Chris S. Kim	4e40e1d222	Handle server addresses update as client	2 years ago
Chris S. Kim	584d3409c4	Send server addresses on update from server	2 years ago
Chris S. Kim	c9d8ad3939	Add new subscription for server addresses	2 years ago
Chris S. Kim	028b87d51f	Cleanup unused logger	2 years ago
freddygv	1031ffc3c7	Re-validate existing secrets at state store ... Previously establishment and pending secrets were only checked at the RPC layer. However, given that these are Check-and-Set transactions we should ensure that the given secrets are still valid when persisting a secret exchange or promotion. Otherwise it would be possible for concurrent requests to overwrite each other.	2 years ago
freddygv	c04515a844	Use proto message for each secrets write op ... Previously there was a field indicating the operation that triggered a secrets write. Now there is a message for each operation and it contains the secret ID being persisted.	2 years ago
freddygv	60d6e28c97	Pass explicit signal with op for secrets write ... Previously the updates to the peering secrets UUID table relied on inferring what action triggered the update based on a reconciliation against the existing secrets. Instead we now explicitly require the operation to be given so that the inference isn't necessary. This makes the UUID table logic easier to reason about and fixes some related bugs. There is also an update so that the peering secrets get handled on snapshots/restores.	2 years ago
Freddy	42996411cc	Various peering fixes (#13979) ... * Avoid logging StreamSecretID * Wrap additional errors in stream handler * Fix flakiness in leader test and rename servers for clarity. There was a race condition where the peering was being deleted in the test before the stream was active. Now the test waits for the stream to be connected on both sides before deleting the associated peering. * Run flaky test serially	2 years ago
Matt Keeler	f74d0cef7a	Implement/Utilize secrets for Peering Replication Stream (#13977)	2 years ago
Luke Kysow	95096e2c03	peering: retry establishing connection more quickly on certain errors (#13938) ... When we receive a FailedPrecondition error, retry that more quickly because we expect it will resolve shortly. This is particularly important in the context of Consul servers behind a load balancer because when establishing a connection we have to retry until we randomly land on a leader node. The default retry backoff goes from 2s, 4s, 8s, etc. which can result in very long delays quite quickly. Instead, this backoff retries in 8ms five times, then goes exponentially from there: 16ms, 32ms, ... up to a max of 8152ms.	2 years ago
Luke Kysow	465a9801e1	Merge pull request #13924 from hashicorp/lkysow/util-metric-peering ... peering: don't track imported services/nodes in usage	2 years ago
Luke Kysow	021b00e321	Remove duplicate comment	2 years ago
Luke Kysow	8c5b70d227	Rename receive to recv in tracker (#13896) ... Because it's shorter	2 years ago
Luke Kysow	3530d3782d	peering: read endpoints can now return failing status (#13849) ... Track streams that have been disconnected due to an error and set their statuses to failing.	2 years ago
alex	927cee692b	peering: emit exported services count metric (#13811) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
Luke Kysow	0c87be0845	peering: Add heartbeating to peering streams (#13806) ... * Add heartbeating to peering streams	2 years ago
Luke Kysow	c411e6b326	Add send mutex to protect against concurrent sends (#13805)	2 years ago
alex	de5a991d8c	peering: refactor reconcile, cleanup (#13795) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
Luke Kysow	e8d965e56f	peerstream: set keepalive enforcement to 15s (#13796) ... The client is set to send keepalive pings every 30s. The server keepalive enforcement must be set to a number less than that, otherwise it will disconnect clients for sending pings too often. MinTime governs the minimum amount of time between pings.	2 years ago
alex	a9ae2ff4fa	peering: track exported services (#13784) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
R.B. Boyer	cd513aeead	peerstream: require a resource subscription to receive updates of that type (#13767) ... This mimics xDS's discovery protocol where you must request a resource explicitly for the exporting side to send those events to you. As part of this I aligned the overall ResourceURL with the TypeURL that gets embedded into the encoded protobuf Any construct. The CheckServiceNodes is now wrapped in a better named "ExportedService" struct now.	2 years ago
R.B. Boyer	c737301093	peerstream: fix test assertions (#13780)	2 years ago
Luke Kysow	ca3d7c964c	peerstream: dialer should reconnect when stream closes (#13745) ... * peerstream: dialer should reconnect when stream closes If the stream is closed unexpectedly (i.e. when we haven't received a terminated message), the dialer should attempt to re-establish the stream. Previously, the `HandleStream` would return `nil` when the stream was closed. The caller then assumed the stream was terminated on purpose and so didn't reconnect when instead it was stopped unexpectedly and the dialer should have attempted to reconnect.	2 years ago
R.B. Boyer	bb4d4040fb	server: ensure peer replication can successfully use TLS over external gRPC (#13733) ... Ensure that the peer stream replication rpc can successfully be used with TLS activated. Also: - If key material is configured for the gRPC port but HTTPS is not enabled now TLS will still be activated for the gRPC port. - peerstream replication stream opened by the establishing-side will now ignore grpc.WithBlock so that TLS errors will bubble up instead of being awkwardly delayed or suppressed	2 years ago
alex	adb5ffa1a6	peering: track imported services (#13718)	2 years ago
Matt Keeler	257f88d4df	Use Node Name for peering healthSnapshot instead of ID (#13773) ... A Node ID is not a required field with Consul’s data model. Therefore we cannot reliably expect all uses to have it. However the node name is required and must be unique so its equally as good of a key for the internal healthSnapshot node tracking.	2 years ago
Chris S. Kim	b4ffa9ae0c	Scrub VirtualIPs before exporting	2 years ago
Dan Upton	b9e525d689	grpc: rename public/private directories to external/internal (#13721) ... Previously, public referred to gRPC services that are both exposed on the dedicated gRPC port and have their definitions in the proto-public directory (so were considered usable by 3rd parties). Whereas private referred to services on the multiplexed server port that are only usable by agents and other servers. Now, we're splitting these definitions, such that external/internal refers to the port and public/private refers to whether they can be used by 3rd parties. This is necessary because the peering replication API needs to be exposed on the dedicated port, but is not (yet) suitable for use by 3rd parties.	2 years ago