consul

Commit Graph

Author	SHA1	Message	Date
Semir Patel	3c937f3d25	[BACKPORT] 1.14.x manual backport of OSS->CE branch (#18556 )	1 year ago
hc-github-team-consul-core	148a79fe93	Backport of [OSS] Fix initial_fetch_timeout to wait for all xDS resources into release/1.14.x (#18066 ) * fix(connect): set initial_fetch_time to wait indefinitely * changelog --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>	1 year ago
hc-github-team-consul-core	fb19faa5a5	Backport of [OSS] gRPC Blocking Queries into release/1.14.x (#17440 ) * backport of feat: initial grpc blocking queries (`11b1dd2`) * backport of changelog and docs update (`5ceef20`) --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>	2 years ago
hc-github-team-consul-core	1db92aff42	Backport of Fix multiple issues related to proxycfg health queries. into release/1.14.x (#17267 ) * backport of commit `464f79a5d3` * backport of commit `ff4ba957b6` * backport of commit `ab3e904470` --------- Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>	2 years ago
Eric Haberkorn	a4d98f3e81	proxycfg: ensure that an irrecoverable error in proxycfg closes the xds session and triggers a replacement proxycfg watcher (#16497 ) (#17030 ) Receiving an "acl not found" error from an RPC in the agent cache and the streaming/event components will cause any request loops to cease under the assumption that they will never work again if the token was destroyed. This prevents log spam (#14144, #9738). Unfortunately due to things like: - authz requests going to stale servers that may not have witnessed the token creation yet - authz requests in a secondary datacenter happening before the tokens get replicated to that datacenter - authz requests from a primary TO a secondary datacenter happening before the tokens get replicated to that datacenter The caller will get an "acl not found" before the token exists, rather than just after. The machinery added above in the linked PRs will kick in and prevent the request loop from looping around again once the tokens actually exist. For `consul-dataplane` usages, where xDS is served by the Consul servers rather than the clients ultimately this is not a problem because in that scenario the `agent/proxycfg` machinery is on-demand and launched by a new xDS stream needing data for a specific service in the catalog. If the watching goroutines are terminated it ripples down and terminates the xDS stream, which CDP will eventually re-establish and restart everything. For Consul client usages, the `agent/proxycfg` machinery is ahead-of-time launched at service registration time (called "local" in some of the proxycfg machinery) so when the xDS stream comes in the data is already ready to go. If the watching goroutines terminate it should terminate the xDS stream, but there's no mechanism to re-spawn the watching goroutines. If the xDS stream reconnects it will see no `ConfigSnapshot` and will not get one again until the client agent is restarted, or the service is re-registered with something changed in it. This PR fixes a few things in the machinery: - there was an inadvertent deadlock in fetching snapshot from the proxycfg machinery by xDS, such that when the watching goroutine terminated the snapshots would never be fetched. This caused some of the xDS machinery to get indefinitely paused and not finish the teardown properly. - Every 30s we now attempt to re-insert all locally registered services into the proxycfg machinery. - When services are re-inserted into the proxycfg machinery we special case "dead" ones such that we unilaterally replace them rather that doing that conditionally.	2 years ago
Derek Menteer	35376c1ad0	Backport: Change partition for peers in discovery chain targets (#16772 ) This commit swaps the partition field to the local partition for discovery chains targeting peers. Prior to this change, peer upstreams would always use a value of default regardless of which partition they exist in. This caused several issues in xds / proxycfg because of id mismatches. Some prior fixes were made to deal with one-off id mismatches that this PR also cleans up, since they are no longer needed.	2 years ago
Nitya Dhanushkodi	d6d4f94d0a	[release/1.14.x] peering: peering partition failover fixes (#16693 ) add local source partition for peered upstreams	2 years ago
Andrew Stucki	673653caa0	Manual Backport of Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable into release/1.14.x (#16561 ) * Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable * Regenerate golden files * Add RequestTimeout field * Add changelog entry	2 years ago
Andrew Stucki	89cee200ec	Manual Backport of Fix issue where terminating gateway service resolvers weren't properly cleaned up into release/1.14.x (#16558 ) * Fix issue where terminating gateway service resolvers weren't properly cleaned up * Add integration test for cleaning up resolvers * Add changelog entry * Use state test and drop integration test	2 years ago
hc-github-team-consul-core	0442ef1296	Backport of Fix mesh gateways incorrectly matching peer locality. into release/1.14.x (#16286 ) * backport of commit `9bc7f6280a` * backport of commit `0c48c9a1e8` --------- Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>	2 years ago
Derek Menteer	6717f6c239	Fix nil-pointer panics from proxycfg package. (#16280 ) Prior to this PR, servers / agents would panic and crash if an ingress or api gateway were configured to use a discovery chain that both: 1. Referenced a peered service 2. Had a mesh gateway mode of local This could occur, because code for handling upstream watches was shared between both connect-proxy and the gateways. As a short-term fix, this PR ensures that the maps are always initialized for these gateway services. This PR also wraps the proxycfg execution and service registration calls with recover statements to ensure that future issues like this do not put the server into an unrecoverable state.	2 years ago
Derek Menteer	0379dabe4e	Fix configuration merging for implicit tproxy upstreams. (#16000 ) (#16007 ) Fix configuration merging for implicit tproxy upstreams. Change the merging logic so that the wildcard upstream has correct proxy-defaults and service-defaults values combined into it. It did not previously merge all fields, and the wildcard upstream did not exist unless service-defaults existed (it ignored proxy-defaults, essentially). Change the way we fetch upstream configuration in the xDS layer so that it falls back to the wildcard when no matching upstream is found. This is what allows implicit peer upstreams to have the correct "merged" config. Change proxycfg to always watch local mesh gateway endpoints whenever a peer upstream is found. This simplifies the logic so that we do not have to inspect the "merged" configuration on peer upstreams to extract the mesh gateway mode.	2 years ago
hc-github-team-consul-core	64eafb312e	backport of commit `699d4b0bac` (#15889 ) Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>	2 years ago
hc-github-team-consul-core	8843452d43	Backport of Fix DialedDirectly configuration for Consul dataplane. into release/1.14.x (#15779 ) * backport of commit `1824f4428a` * backport of commit `b9b929daaa` * backport of commit `ea07eeb188` Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>	2 years ago
hc-github-team-consul-core	17d6705c5a	Backport of Fix local mesh gateway with peering discovery chains. into release/1.14.x (#15715 ) * backport of commit `8aff79edfe` * backport of commit `3ed9331e48` Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>	2 years ago
hc-github-team-consul-core	7c0eec4c1f	Add support for configuring Envoys route idle_timeout (#14340 ) (#15611 ) * Add idleTimeout Co-authored-by: James Oulman <oulman@users.noreply.github.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
hc-github-team-consul-core	5843befad2	Backport of Log Warnings When Peering With Mesh Gateway Mode None into release/1.14.x (#15310 ) This pull request was automerged via backport-assistant	2 years ago
hc-github-team-consul-core	11aaa9bcb3	Backport of Fix mesh gateway configuration with proxy-defaults into release/1.14.x (#15309 ) This pull request was automerged via backport-assistant	2 years ago
hc-github-team-consul-core	c896ae06bf	Backport of Bring back parameter ServerExternalAddresses in GenerateToken endpoint into release/1.14.x (#15296 ) * backport of commit `4199a7c30a` * backport of commit `53be93fb90` * backport of commit `1b0ce4e96c` * backport of commit `1b4ee792cb` * backport of commit `98f3d5aa51` * backport of commit `1f3b30b768` * Fix gomod for merging. Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>	2 years ago
hc-github-team-consul-core	0f05c4741e	Backport of fix(mesh-gateway): remove deregistered service from mesh gateway into release/1.14.x (#15284 ) * backport of commit `8083bba320` Co-authored-by: cskh <hui.kang@hashicorp.com>	2 years ago
hc-github-team-consul-core	8071715461	backport of commit `7436a3feb7` (#15280 ) This pull request was automerged via backport-assistant	2 years ago
hc-github-team-consul-core	dbd8fa042b	backport of commit `3496946787` (#15275 ) This pull request was automerged via backport-assistant	2 years ago
hc-github-team-consul-core	35fb3cb433	Backport of Update go version to 1.19 into release/1.14.x (#15139 ) This pull request was automerged via backport-assistant	2 years ago
Dan Upton	f8b4b41205	proxycfg: fix goroutine leak when service is re-registered (#14988 ) Fixes a bug where we'd leak a goroutine in state.run when the given context was canceled while there was a pending update.	2 years ago
Dan Upton	328e3ff563	proxycfg: rate-limit delivery of config snapshots (#14960 ) Adds a user-configurable rate limiter to proxycfg snapshot delivery, with a default limit of 250 updates per second. This addresses a problem observed in our load testing of Consul Dataplane where updating a "global" resource such as a wildcard intention or the proxy-defaults config entry could starve the Raft or Memberlist goroutines of CPU time, causing general cluster instability.	2 years ago
Dan Upton	e6b55d1d81	perf: remove expensive reflection from xDS hot path (#14934 ) Replaces the reflection-based implementation of proxycfg's ConfigSnapshot.Clone with code generated by deep-copy. While load testing server-based xDS (for consul-dataplane) we discovered this method is extremely expensive. The ConfigSnapshot struct, directly or indirectly, contains a copy of many of the structs in the agent/structs package, which creates a large graph for copystructure.Copy to traverse at runtime, on every proxy reconfiguration.	2 years ago
freddygv	c77123a2aa	Use split var in tests	2 years ago
freddygv	bf51021c07	Use split wildcard partition name This way OSS avoids passing a non-empty label, which will be rejected in OSS consul.	2 years ago
Riddhi Shah	345191a0df	Service http checks data source for agentless proxies (#14924 ) Adds another datasource for proxycfg.HTTPChecks, for use on server agents. Typically these checks are performed by local client agents and there is no equivalent of this in agentless (where servers configure consul-dataplane proxies). Hence, the data source is mostly a no-op on servers but in the case where the service is present within the local state, it delegates to the cache data source.	2 years ago
boruszak	38b1a515f1	Mergimg	2 years ago
James Oulman	b8bd7a3058	Configure Envoy alpn_protocols based on service protocol (#14356 ) * Configure Envoy alpn_protocols based on service protocol * define alpnProtocols in a more standard way * http2 protocol should be h2 only * formatting * add test for getAlpnProtocol() * create changelog entry * change scope is connect-proxy * ignore errors on ParseProxyConfig; fixes linter * add tests for grpc and http2 public listeners * remove newlines from PR * Add alpn_protocol configuration for ingress gateway * Guard against nil tlsContext * add ingress gateway w/ TLS tests for gRPC and HTTP2 * getAlpnProtocols: add TCP protocol test * add tests for ingress gateway with grpc/http2 and per-listener TLS config * add tests for ingress gateway with grpc/http2 and per-listener TLS config * add Gateway level TLS config with mixed protocol listeners to validate ALPN * update changelog to include ingress-gateway * add http/1.1 to http2 ALPN * go fmt * fix test on custom-trace-listener	2 years ago
freddygv	5f97223822	Simplify mgw watch mgmt	2 years ago
DanStough	77ab28c5c7	feat: xDS updates for peerings control plane through mesh gw	2 years ago
Eric Haberkorn	1633cf20ea	Make the mesh gateway changes to allow `local` mode for cluster peering data plane traffic (#14817 ) Make the mesh gateway changes to allow `local` mode for cluster peering data plane traffic	2 years ago
Derek Menteer	a279d2d329	Fix explicit tproxy listeners with discovery chains. (#14751 ) Fix explicit tproxy listeners with discovery chains.	2 years ago
Alex Oskotsky	13da2c5fad	Add the ability to retry on reset connection to service-routers (#12890 )	2 years ago
freddygv	b15d41534f	Update xds generation for peering over mesh gws This commit adds the xDS resources needed for INBOUND traffic from peer clusters: - 1 filter chain for all inbound peering requests. - 1 cluster for all inbound peering requests. - 1 endpoint per voting server with the gRPC TLS port configured. There is one filter chain and cluster because unlike with WAN federation, peer clusters will not attempt to dial individual servers. Peer clusters will only dial the local mesh gateway addresses.	2 years ago
Ashwin Venkatesh	4ba260958c	bug: watch local mesh gateways in non-default partitions with agentless (#14799 )	2 years ago
cskh	69f40df548	feat(ingress gateway: support configuring limits in ingress-gateway c… (#14749 ) * feat(ingress gateway: support configuring limits in ingress-gateway config entry - a new Defaults field with max_connections, max_pending_connections, max_requests is added to ingress gateway config entry - new field max_connections, max_pending_connections, max_requests in individual services to overwrite the value in Default - added unit test and integration test - updated doc Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	2 years ago
Eric Haberkorn	6570d5f004	Enable outbound peered requests to go through local mesh gateway (#14763 )	2 years ago
Nick Ethier	1c1b0994b8	add HCP integration component (#14723 ) * add HCP integration * lint: use non-deprecated logging interface	2 years ago
freddygv	d818d7b096	Manage local server watches depending on mesh cfg Routing peering control plane traffic through mesh gateways can be enabled or disabled at runtime with the mesh config entry. This commit updates proxycfg to add or cancel watches for local servers depending on this central config. Note that WAN federation over mesh gateways is determined by a service metadata flag, and any updates to the gateway service registration will force the creation of a new snapshot. If enabled, WAN-fed over mesh gateways will trigger a local server watch on initialize(). Because of this we will only add/remove server watches if WAN federation over mesh gateways is disabled.	2 years ago
Eric Haberkorn	aa8268e50c	Implement Cluster Peering Redirects (#14445 ) implement cluster peering redirects	2 years ago
Daniel Upton	8c46e48e0d	proxycfg-glue: server-local implementation of IntentionUpstreamsDestination This is the OSS portion of enterprise PR 2463. Generalises the serverIntentionUpstreams type to support matching on a service or destination.	2 years ago
Daniel Upton	f8dba7e9ac	proxycfg-glue: server-local implementation of InternalServiceDump This is the OSS portion of enterprise PR 2489. This PR introduces a server-local implementation of the proxycfg.InternalServiceDump interface that sources data from a blocking query against the server's state store. For simplicity, it only implements the subset of the Internal.ServiceDump RPC handler actually used by proxycfg - as such the result type has been changed to IndexedCheckServiceNodes to avoid confusion.	2 years ago
Eric Haberkorn	3726a0ab7a	Finish up cluster peering failover (#14396 )	2 years ago
Daniel Upton	13c04a13af	proxycfg: terminate stream on irrecoverable errors This is the OSS portion of enterprise PR 2339. It improves our handling of "irrecoverable" errors in proxycfg data sources. The canonical example of this is what happens when the ACL token presented by Envoy is deleted/revoked. Previously, the stream would get "stuck" until the xDS server re-checked the token (after 5 minutes) and terminated the stream. Materializers would also sit burning resources retrying something that could never succeed. Now, it is possible for data sources to mark errors as "terminal" which causes the xDS stream to be closed immediately. Similarly, the submatview.Store will evict materializers when it observes they have encountered such an error.	2 years ago
Eric Haberkorn	58901ad7df	Cluster peering failover disco chain changes (#14296 )	2 years ago
Dhia Ayachi	6fd65a4a45	Tgtwy egress HTTP support (#13953 ) * add golden files * add support to http in tgateway egress destination * fix slice sorting to include both address and port when using server_names * fix listener loop for http destination * fix routes to generate a route per port and a virtualhost per port-address combination * sort virtual hosts list to have a stable order * extract redundant serviceNode	2 years ago
Dhia Ayachi	256694b603	inject gateway addons to destination clusters (#13951 )	2 years ago

1 2 3 4 5 ...

299 Commits (98dc578e04b75e428c34a43f787955d2f6d98410)