214 Commits (499fbdabaf26d43281d7f4a8f1fe8bdbad2b262d)

Author	SHA1	Message	Date
Paul Banks	20d0bf81f7	Revert abandonned changes to proxycfg for Ent test consistency	3 years ago
Paul Banks	659321d008	Handle namespaces in route names correctly; add tests for enterprise	3 years ago
Paul Banks	ccbda0c285	Update proxycfg to hold more ingress config state	3 years ago
Paul Banks	4e39f03d5b	Add ingress-gateway config for SDS	3 years ago
freddygv	49248a0802	Fixup proxycfg tproxy case	3 years ago
freddygv	95a6db9cfa	Account for partitions in ixn match/decision	3 years ago
freddygv	3f3a61c6e1	Fixup manager tests	3 years ago
freddygv	77681b9f6c	Pass partition to intention match query	3 years ago
Paul Banks	e22cc9c53a	Header manip for split legs plumbing	3 years ago
Paul Banks	83fc8723a3	Header manip for service-router plumbed through	3 years ago
Paul Banks	f439dfc04f	Ingress gateway header manip plumbing	3 years ago
Dhia Ayachi	bc0e4f2f46	partition dicovery chains (#10983) ... * partition dicovery chains * fix default partition for OSS	3 years ago
Dhia Ayachi	09197c989c	add partition to SNI when partition is non default (#10917)	3 years ago
freddygv	f52bd80f6d	Update comment for test function	3 years ago
freddygv	af52d21884	Update prepared query cluster SAN validation ... Previously SAN validation for prepared queries was broken because we validated against the name, namespace, and datacenter for prepared queries. However, prepared queries can target: - Services with a name that isn't their own - Services in multiple datacenters This means that the SpiffeID to validate needs to be based on the prepared query endpoints, and not the prepared query's upstream definition. This commit updates prepared query clusters to account for that.	3 years ago
freddygv	85878685b7	Fixup proxy config test fixtures ... - The TestNodeService helper created services with the fixed name "web", and now that name is overridable. - The discovery chain snapshot didn't have prepared query endpoints so the endpoints tests were missing data for prepared queries	3 years ago
Dhia Ayachi	1950ebbe1f	oss portion of ent #1069 (#10883)	3 years ago
R.B. Boyer	097e1645e3	agent: ensure that most agent behavior correctly respects partition configuration (#10880)	3 years ago
Daniel Nephin	0575498d0d	proxycfg: Lookup the agent token as a default ... When no ACL token is provided with the service registration.	3 years ago
Daniel Nephin	b313f495b8	proxycfg: Add a test to show the bug ... When a token is not provided at registration, the agent token is not being used.	3 years ago
Freddy	19f6e1ca31	Log the correlation ID when blocking queries fire (#10689) ... Knowing that blocking queries are firing does not provide much information on its own. If we know the correlation IDs we can piece together which parts of the snapshot have been populated. Some of these responses might be empty from the blocking query timing out. But if they're returning quickly I think we can reasonably assume they contain data.	3 years ago
R.B. Boyer	188e8dc51f	agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning (#10669)	3 years ago
freddygv	b4c5c58c9b	Add TODOs about partition handling	3 years ago
freddygv	47da00d3c7	Validate SANs for passthrough clusters and failovers	3 years ago
Daniel Nephin	10051cf6d3	proxycfg: remove unused method ... This method was accidentally re-introduced in an earlier rebase. It was removed in ed1082510d as part of the tproxy work.	4 years ago
Daniel Nephin	6bc5255028	proxycfg: move each handler into a seprate file ... There is no interaction between these handlers, so splitting them into separate files makes it easier to discover the full implementation of each kindHandler.	4 years ago
Daniel Nephin	19d3eeff3c	Merge pull request #9489 from hashicorp/dnephin/proxycfg-state-2 ... proxycfg: split state into a handler for each kind	4 years ago
Nitya Dhanushkodi	52043830b4	proxycfg: reference to entry in map should not panic	4 years ago
Daniel Nephin	e738fa3b80	Replace type conversion with embedded structs	4 years ago
Daniel Nephin	32c15d9a88	proxycfg: split state into kind-specific types ... This commit extracts all the kind-specific logic into handler types, and keeps the generic parts on the state struct. This change should make it easier to add new kinds, and see the implementation of each kind more clearly.	4 years ago
Daniel Nephin	cd05df7157	proxycfg: unmethod hostnameEndpoints ... the method receiver can be replaced by the first argument. This will allow us to extract more from the state struct in the future.	4 years ago
Daniel Nephin	97c6ee00d7	Remove duplicate import ... because two PRs crossed paths.	4 years ago
Daniel Nephin	0547d0c046	Merge pull request #9466 from hashicorp/dnephin/proxycfg-state ... proxycfg: prepare state for split by kind	4 years ago
Nitya Dhanushkodi	b8b44419a0	proxycfg: Ensure that endpoints for explicit upstreams in other datacenters are watched in transparent mode (#10391) ... Co-authored-by: Freddy Vallenilla <freddy@hashicorp.com>	4 years ago
Daniel Nephin	016c5611d1	proxycfg: extract two types from state struct ... These two new struct types will allow us to make polymorphic handler for each kind, instad of having all the logic for each proxy kind on the state struct.	4 years ago
Daniel Nephin	9c40aa729f	proxycfg: pass context around where it is needed ... context.Context should never be stored on a struct (as it says in the godoc) because it is easy to to end up with the wrong context when it is stored. Also see https://blog.golang.org/context-and-structs This change is also in preparation for splitting state into kind-specific handlers so that the implementation of each kind is grouped together.	4 years ago
Freddy	429f9d8bb8	Add flag for transparent proxies to dial individual instances (#10329)	4 years ago
freddygv	c73703c08b	Ensure entmeta is encoded in test correlationID	4 years ago
Daniel Nephin	347f3d2128	Merge pull request #10155 from hashicorp/dnephin/config-entry-remove-fields ... config-entry: remove Kind and Name field from Mesh config entry	4 years ago
Mark Anderson	6be9cebad0	Add tests for xds/listeners ... Signed-off-by: Mark Anderson <manderson@hashicorp.com>	4 years ago
Mark Anderson	06f0f79218	Continue working through proxy and agent ... Rework/listeners, rename makeListener Refactor, tests pass Signed-off-by: Mark Anderson <manderson@hashicorp.com>	4 years ago
Freddy	ed1082510d	Fixup discovery chain handling in transparent mode (#10168) ... Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> Previously we would associate the address of a discovery chain target with the discovery chain's filter chain. This was broken for a few reasons: - If the upstream is a virtual service, the client proxy has no way of dialing it because virtual services are not targets of their discovery chains. The targets are distinct services. This is addressed by watching the endpoints of all upstream services, not just their discovery chain targets. - If multiple discovery chains resolve to the same target, that would lead to multiple filter chains attempting to match on the target's virtual IP. This is addressed by only matching on the upstream's virtual IP. NOTE: this implementation requires an intention to the redirecting virtual service and not just to the final destination. This is how we can know that the virtual service is an upstream to watch. A later PR will look into traversing discovery chains when computing upstreams so that intentions are only required to the discovery chain targets.	4 years ago
Daniel Nephin	62efaaab21	config-entry: remove Kind and Name field from Mesh config entry ... No config entry needs a Kind field. It is only used to determine the Go type to target. As we introduce new config entries (like this one) we can remove the kind field and have the GetKind method return the single supported value. In this case (similar to proxy-defaults) the Name field is also unnecessary. We always use the same value. So we can omit the name field entirely.	4 years ago
R.B. Boyer	71d45a3460	Support Incremental xDS mode (#9855) ... This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged. Union of all commit messages follows to give an overarching summary: xds: exclusively support incremental xDS when using xDS v3 Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail. Work around a strange older envoy behavior involving empty CDS responses over incremental xDS. xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead. Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client. xds: pull out checkStreamACLs method in advance of a later commit xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty. This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10. xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time	4 years ago
Freddy	078c40425f	Rename "cluster" config entry to "mesh" (#10127) ... This config entry is being renamed primarily because in k8s the name cluster could be confusing given that the config entry applies across federated datacenters. Additionally, this config entry will only apply to Consul as a service mesh, so the more generic "cluster" name is not needed.	4 years ago
Daniel Nephin	2a26085b2c	connect: do not set QuerySource.Node ... Setting this field to a value is equivalent to using the 'near' query paramter. The intent is to sort the results by proximity to the node requesting them. However with connect we send the results to envoy, which doesn't care about the order, so setting this field is increasing the work performed for no gain. It is necessary to unset this field now because we would like connect to use streaming, but streaming does not support sorting by proximity.	4 years ago
Freddy	439a7fce2d	Split Upstream.Identifier() so non-empty namespace is always prepended in ent (#10031)	4 years ago
freddygv	8857195437	Fixup wildcard ent assertion	4 years ago
freddygv	7bd51ff536	Replace TransparentProxy bool with ProxyMode ... This PR replaces the original boolean used to configure transparent proxy mode. It was replaced with a string mode that can be set to: - "": Empty string is the default for when the setting should be defaulted from other configuration like config entries. - "direct": Direct mode is how applications originally opted into the mesh. Proxy listeners need to be dialed directly. - "transparent": Transparent mode enables configuring Envoy as a transparent proxy. Traffic must be captured and redirected to the inbound and outbound listeners. This PR also adds a struct for transparent proxy specific configuration. Initially this is not stored as a pointer. Will revisit that decision before GA.	4 years ago
freddygv	b21224a4c8	PR comments	4 years ago
freddygv	49a4a78fd5	Ensure mesh gateway mode override is set for upstreams for intentions	4 years ago
freddygv	5140c3e51f	Finish resolving upstream defaults in proxycfg	4 years ago
R.B. Boyer	499fee73b3	connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973) ... This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.	4 years ago
freddygv	098b9af901	Fixup enterprise tests from tproxy changes	4 years ago
freddygv	eb1e0a1751	Cancel watch on all errors	4 years ago
freddygv	f4f45af6d0	Merge master and fix upstream config protocol defaulting	4 years ago
freddygv	0da8702f34	PR comments	4 years ago
freddygv	a54d6a9010	Update proxycfg for transparent proxy	4 years ago
Daniel Nephin	f40b76af2d	proxycfg: use rpcclient/health.Client instead of passing around cache name ... This should allow us to swap out the implementation with something other than `agent/cache` without making further code changes.	4 years ago
Daniel Nephin	906834ce8e	proxycfg: Use streaming in connect state	4 years ago
Freddy	82c269a7c5	Avoid potential proxycfg/xDS deadlock using non-blocking send	4 years ago
freddygv	ec5f75776b	Update comments on avoiding proxycfg deadlock	4 years ago
R.B. Boyer	43193a35c6	xds: prevent LDS flaps in mesh gateways due to unstable datacenter lists (#9651) ... Also fix a similar issue in Terminating Gateways that was masked by an overzealous test.	4 years ago
freddygv	6e443e5536	Retry send after timer fires, in case no updates occur	4 years ago
freddygv	95e7641faa	Update proxycfg logging, labels were already attached	4 years ago
freddygv	5ba14ad41d	Add trace logs to proxycfg state runner and xds srv	4 years ago
freddygv	37190c0d0d	Avoid potential deadlock using non-blocking send ... Deadlock scenario: 1. Due to scheduling, the state runner sends one snapshot into snapCh and then attempts to send a second. The first send succeeds because the channel is buffered, but the second blocks. 2. Separately, Manager.Watch is called by the xDS server after getting a discovery request from Envoy. This function acquires the manager lock and then blocks on receiving the CurrentSnapshot from the state runner. 3. Separately, there is a Manager goroutine that reads the snapshots from the channel in step 1. These reads are done to notify proxy watchers, but they require holding the manager lock. This goroutine goes to acquire that lock, but can't because it is held by step 2. Now, the goroutine from step 3 is waiting on the one from step 2 to release the lock. The goroutine from step 2 won't release the lock until the goroutine in step 1 advances. But the goroutine in step 1 is waiting for the one in step 3. Deadlock. By making this send non-blocking step 1 above can proceed. The coalesce timer will be reset and a new valid snapshot will be delivered after it elapses or when one is requested by xDS.	4 years ago
Daniel Nephin	b9e60c0775	testing: skip slow tests with -short ... Add a skip condition to all tests slower than 100ms. This change was made using `gotestsum tool slowest` with data from the last 3 CI runs of master. See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests With this change: ``` $ time go test -count=1 -short ./agent ok github.com/hashicorp/consul/agent 0.743s real 0m4.791s $ time go test -count=1 -short ./agent/consul ok github.com/hashicorp/consul/agent/consul 4.229s real 0m8.769s ```	4 years ago
freddygv	856d5a25ee	Fix text type assertion	4 years ago
freddygv	7fd518ff1d	Merge master	4 years ago
freddygv	87541ab80a	Fix type assertion	4 years ago
freddygv	768dbaa68d	Add session flag to cookie config	4 years ago
freddygv	eab90ea9fa	Revert EnvoyConfig nesting	4 years ago
freddygv	30ba080d25	Add explicit protocol overrides in tgw xds test cases	4 years ago
freddygv	f81fe6a1a1	Remove LB infix and move injection to xds	4 years ago
freddygv	63f79e5f9b	Restructure structs and other PR comments	4 years ago
freddygv	28d0602fc1	Pass LB config to Envoy via xDS	4 years ago
R.B. Boyer	74d5df7c7a	xds: use envoy's rbac filter to handle intentions entirely within envoy (#8569)	4 years ago
Matt Keeler	be01c4241d	Default Cache rate limiting options in New ... Also get rid of the TestCache helper which was where these defaults were happening previously.	4 years ago
Pierre Souchay	505de6dc29	Added ratelimit to handle throtling cache (#8226) ... This implements a solution for #7863 It does: Add a new config cache.entry_fetch_rate to limit the number of calls/s for a given cache entry, default value = rate.Inf Add cache.entry_fetch_max_burst size of rate limit (default value = 2) The new configuration now supports the following syntax for instance to allow 1 query every 3s: command line HCL: -hcl 'cache = { entry_fetch_rate = 0.333}' in JSON { "cache": { "entry_fetch_rate": 0.333 } }	4 years ago
Matt Keeler	12acdd7481	Disable background cache refresh for Connect Leaf Certs ... The rationale behind removing them is that all of our own code (xDS, builtin connect proxy) use the cache notification mechanism. This ensures that the blocking fetch behind the scenes is always executing. Therefore the only way you might go to get a certificate and have to wait is when 1) the request has never been made for that cert before or 2) you are using the v1/agent/connect/ca/leaf API for retrieving the cert yourself. In the first case, the refresh change doesn’t alter the behavior. In the second case, it can be mitigated by using blocking queries with that API which just like normal cache notification mechanism will cause the blocking fetch to be initiated and to get leaf certs as soon as needed. If you are not using blocking queries, or Envoy/xDS, or the builtin connect proxy but are retrieving the certs yourself then the HTTP endpoint might take a little longer to respond. This also renames the RefreshTimeout field on the register options to QueryTimeout to more accurately reflect that it is used for any type that supports blocking queries.	4 years ago
Daniel Nephin	010a609912	Fix a bunch of unparam lint issues	5 years ago
Freddy	5baa7b1b04	Always return a gateway cluster (#8158)	5 years ago
Daniel Nephin	5afcf5c1bc	Merge pull request #8034 from hashicorp/dnephin/add-linter-staticcheck-4 ... ci: enable SA4006 staticcheck check and add ineffassign	5 years ago
Daniel Nephin	068b43df90	Enable gofmt simplify ... Code changes done automatically with 'gofmt -s -w'	5 years ago
Daniel Nephin	cb050b280c	ci: enable SA4006 staticcheck check ... And fix the 'value not used' issues. Many of these are not bugs, but a few are tests not checking errors, and one appears to be a missed error in non-test code.	5 years ago
freddygv	19e3954603	Move compound service names to use ServiceName type	5 years ago
Freddy	166a8b2a58	Only pass one hostname via EDS and prefer healthy ones (#8084) ... Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Currently when passing hostname clusters to Envoy, we set each service instance registered with Consul as an LbEndpoint for the cluster. However, Envoy can only handle one per cluster: [2020-06-04 18:32:34.094][1][warning][config] [source/common/config/grpc_subscription_impl.cc:87] gRPC config for type.googleapis.com/envoy.api.v2.Cluster rejected: Error adding/updating cluster(s) dc2.internal.ddd90499-9b47-91c5-4616-c0cbf0fc358a.consul: LOGICAL_DNS clusters must have a single locality_lb_endpoint and a single lb_endpoint, server.dc2.consul: LOGICAL_DNS clusters must have a single locality_lb_endpoint and a single lb_endpoint Envoy is currently handling this gracefully by only picking one of the endpoints. However, we should avoid passing multiple to avoid these warning logs. This PR: * Ensures we only pass one endpoint, which is tied to one service instance. * We prefer sending an endpoint which is marked as Healthy by Consul. * If no endpoints are healthy we emit a warning and skip the cluster. * If multiple unique hostnames are spread across service instances we emit a warning and let the user know which will be resolved.	5 years ago
Freddy	9ed325ba8b	Enable gateways to resolve hostnames to IPv4 addresses (#7999) ... The DNS resolution will be handled by Envoy and defaults to LOGICAL_DNS. This discovery type can be overridden on a per-gateway basis with the envoy_dns_discovery_type Gateway Option. If a service contains an instance with a hostname as an address we set the Envoy cluster to use DNS as the discovery type rather than EDS. Since both mesh gateways and terminating gateways route to clusters using SNI, whenever there is a mix of hostnames and IP addresses associated with a service we use the hostname + CDS rather than the IPs + EDS. Note that we detect hostnames by attempting to parse the service instance's address as an IP. If it is not a valid IP we assume it is a hostname.	5 years ago
Daniel Nephin	c88fae0aac	ci: Add staticcheck and fix most errors ... Three of the checks are temporarily disabled to limit the size of the diff, and allow us to enable all the other checks in CI. In a follow up we can fix the issues reported by the other checks one at a time, and enable them.	5 years ago
Kyle Havlovitz	b14696e32a	Standardize support for Tagged and BindAddresses in Ingress Gateways (#7924) ... * Standardize support for Tagged and BindAddresses in Ingress Gateways This updates the TaggedAddresses and BindAddresses behavior for Ingress to match Mesh/Terminating gateways. The `consul connect envoy` command now also allows passing an address without a port for tagged/bind addresses. * Update command/connect/envoy/envoy.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * PR comments * Check to see if address is an actual IP address * Update agent/xds/listeners.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * fix whitespace Co-authored-by: Chris Piraino <cpiraino@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com>	5 years ago
Chris Piraino	9d9e23cc44	Add service id context to the proxycfg logger ... This is especially useful when multiple proxies are all querying the same Consul agent.	5 years ago
Kyle Havlovitz	136549205c	Merge pull request #7759 from hashicorp/ingress/tls-hosts ... Add TLS option for Ingress Gateway listeners	5 years ago
Chris Piraino	a0e1f57ac2	Remove development log line	5 years ago
Chris Piraino	26f92e74f6	Compute all valid DNSSANs for ingress gateways ... For DNSSANs we take into account the following and compute the appropriate wildcard values: - source datacenter - namespaces - alt domains	5 years ago
Freddy	c32a4f1ece	Fix up enterprise compatibility for gateways (#7813)	5 years ago
Chris Piraino	0bd5618cb2	Cleanup proxycfg for TLS ... - Use correct enterprise metadata for finding config entry - nil out cancel functions on config snapshot copy - Look at HostsSet when checking validity	5 years ago
Freddy	b069887b2a	Remove timeout and call to Fatal from goroutine (#7797)	5 years ago
Kyle Havlovitz	f14c54e25e	Add TLS option and DNS SAN support to ingress config ... xds: Only set TLS context for ingress listener when requested	5 years ago
Chris Piraino	881760f701	xds: Use only the port number as the configured route name ... This removes duplication of protocol from the stats_prefix	5 years ago