4041 Commits (cbea3d203c44db86a37920c43015e10e66ecfd97)

Author	SHA1	Message	Date
freddygv	cbea3d203c	Fix race of upstreams with same passthrough ip ... Due to timing, a transparent proxy could have two upstreams to dial directly with the same address. For example: - The orders service can dial upstreams shipping and payment directly. - An instance of shipping at address 10.0.0.1 is deregistered. - Payments is scaled up and scheduled to have address 10.0.0.1. - The orders service receives the event for the new payments instance before seeing the deregistration for the shipping instance. At this point two upstreams have the same passthrough address and Envoy will reject the listener configuration. To disambiguate this commit considers the Raft index when storing passthrough addresses. In the example above, 10.0.0.1 would only be associated with the newer payments service instance.	3 years ago
freddygv	659ebc05a9	Ensure passthrough addresses get cleaned up ... Transparent proxies can set up filter chains that allow direct connections to upstream service instances. Services that can be dialed directly are stored in the PassthroughUpstreams map of the proxycfg snapshot. Previously these addresses were not being cleaned up based on new service health data. The list of addresses associated with an upstream service would only ever grow. As services scale up and down, eventually they will have instances assigned to an IP that was previously assigned to a different service. When IP addresses are duplicated across filter chain match rules the listener config will be rejected by Envoy. This commit updates the proxycfg snapshot management so that passthrough addresses can get cleaned up when no longer associated with a given upstream. There is still the possibility of a race condition here where due to timing an address is shared between multiple passthrough upstreams. That concern is mitigated by #12195, but will be further addressed in a follow-up.	3 years ago
freddygv	c31c1158a6	Add failing test ... The updated test fails because passthrough upstream addresses are not being cleaned up.	3 years ago
Daniel Nephin	9b7468f99e	ca/provider: remove ActiveRoot from Provider	3 years ago
Daniel Nephin	c2b9c81a55	ca: update MockProvider for new interface	3 years ago
Daniel Nephin	f05bad4a1d	ca: update GenerateRoot godoc	3 years ago
Daniel Nephin	9a59733b7d	Merge pull request #11663 from hashicorp/dnephin/ca-remove-one-call-to-active-root-2 ... ca: remove second call to Provider.ActiveRoot	3 years ago
Daniel Nephin	db0478265b	Merge pull request #12109 from hashicorp/dnephin/blocking-query-1 ... rpc: make blockingQuery easier to read	3 years ago
Daniel Nephin	7125fec346	Merge pull request #11221 from hashicorp/dnephin/acl-resolver-5 ... acl: extract a backend type for the ACLResolverBackend	3 years ago
Dao Thanh Tung	759dd93544	URL-encode/decode resource names for HTTP API part 3 (#12103)	3 years ago
Daniel Nephin	f9aef8018b	Apply suggestions from code review ... Co-authored-by: Chris S. Kim <ckim@hashicorp.com>	3 years ago
Daniel Nephin	737c0097e0	acl: extract a backend type for the ACLResolverBackend ... This is a small step to isolate the functionality that is used for the ACLResolver from the large Client and Server structs.	3 years ago
R.B. Boyer	d2c0945f52	xds: fix for delta xDS reconnect bug in LDS/CDS (#12174) ... When a wildcard xDS type (LDS/CDS/SRDS) reconnects from a delta xDS stream, prior to envoy `1.19.0` it would populate the `ResourceNamesSubscribe` field with the full list of currently subscribed items, instead of simply omitting it to infer that it wanted everything (which is what wildcard mode means). This upstream issue was filed in envoyproxy/envoy#16063 and fixed in envoyproxy/envoy#16153 which went out in Envoy `1.19.0` and is fixed in later versions (later refactored in envoyproxy/envoy#16855). This PR conditionally forces LDS/CDS to be wildcard-only even when the connected Envoy requests a non-wildcard subscription, but only does so on versions prior to `1.19.0`, as we should not need to do this on later versions. This fixes the failure case as described here: #11833 (comment) Co-authored-by: Huan Wang <fredwanghuan@gmail.com>	3 years ago
Chris S. Kim	bee18f4a1d	Generate bindata_assetfs.go (#12146)	3 years ago
R.B. Boyer	b60d89e7ef	bulk rewrite using this script ... set -euo pipefail unset CDPATH cd "$(dirname "$0")" for f in $(git grep '\brequire := require\.New(' | cut -d':' -f1 | sort -u); do echo "=== require: $f ===" sed -i '/require := require.New(t)/d' $f # require.XXX(blah) but not require.XXX(tblah) or require.XXX(rblah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\([^tr]\)/require.\1(t,\2/g' $f # require.XXX(tblah) but not require.XXX(t, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/require.\1(t,\2/g' $f # require.XXX(rblah) but not require.XXX(r, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/require.\1(t,\2/g' $f gofmt -s -w $f done for f in $(git grep '\bassert := assert\.New(' | cut -d':' -f1 | sort -u); do echo "=== assert: $f ===" sed -i '/assert := assert.New(t)/d' $f # assert.XXX(blah) but not assert.XXX(tblah) or assert.XXX(rblah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\([^tr]\)/assert.\1(t,\2/g' $f # assert.XXX(tblah) but not assert.XXX(t, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/assert.\1(t,\2/g' $f # assert.XXX(rblah) but not assert.XXX(r, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/assert.\1(t,\2/g' $f gofmt -s -w $f done	3 years ago
R.B. Boyer	31f6f55bbe	test: normalize require.New and assert.New syntax	3 years ago
R.B. Boyer	424f3cdd2c	proxycfg: introduce explicit UpstreamID in lieu of bare string (#12125) ... The gist here is that now we use a value-type struct proxycfg.UpstreamID as the map key in ConfigSnapshot maps where we used to use "upstream id-ish" strings. These are internal only and used just for bidirectional trips through the agent cache keyspace (like the discovery chain target struct). For the few places where the upstream id needs to be projected into xDS, that's what (proxycfg.UpstreamID).EnvoyID() is for. This lets us ALWAYS inject the partition and namespace into these things without making stuff like the golden testdata diverge.	3 years ago
Dan Upton	ca3aca92c4	[OSS] Remove remaining references to master (#11827)	3 years ago
VictorBac	31a39c9528	Add GRPC and GRPCUseTLS to api.HealthCheckDefinition (#12108) ... * Add GRPC to HealthCheckDefinition * add GRPC and GRPCUseTLS	3 years ago
Evan Culver	e35dd08a63	connect: Upgrade Envoy 1.20 to 1.20.1 (#11895)	3 years ago
Daniel Nephin	71767f1b3e	rpc: cleanup exit and blocking condition logic in blockingQuery ... Remove some unnecessary comments around query_blocking metric. The only line that needs any comments in the atomic decrement. Cleanup the block and return comments and logic. The old comment about AbandonCh may have been relevant before, but it is expected behaviour now. The logic was simplified by inverting the err condition.	3 years ago
Daniel Nephin	72a733bed8	rpc: extract rpcQueryTimeout method ... This helps keep the logic in blockingQuery more focused. In the future we may have a separate struct for RPC queries which may allow us to move this off of Server.	3 years ago
Daniel Nephin	fd0a9fd4f3	rpc: move the index defaulting to setQueryMeta. ... This safeguard should be safe to apply in general. We are already applying it to non-blocking queries that call blockingQuery, so it should be fine to apply it to others.	3 years ago
Daniel Nephin	4b67d6c18b	rpc: add subtests to blockingQuery test	3 years ago
Daniel Nephin	f92dc11002	rpc: refactor blocking query ... To remove the TODO, and make it more readable. In general this reduces the scope of variables, making them easier to reason about. It also introduces more early returns so that we can see the flow from the structure of the function.	3 years ago
Daniel Nephin	f31e0b8b1a	Merge pull request #11661 from hashicorp/dnephin/ca-remove-one-call-to-active-root ... ca: remove one call to Provider.ActiveRoot	3 years ago
Kyle Havlovitz	0db874c38b	Add virtual IP generation for term gateway backed services	3 years ago
Chris S. Kim	98ea6d1cf1	Fix race with tags (#12041)	3 years ago
Chris S. Kim	a0acf9978f	Fix races in anti-entropy tests (#12028)	3 years ago
Mike Morris	1b1a97e8f9	ingress: allow setting TLS min version and cipher suites in ingress gateway config entries (#11576) ... * xds: refactor ingress listener SDS configuration * xds: update resolveListenerSDS call args in listeners_test * ingress: add TLS min, max and cipher suites to GatewayTLSConfig * xds: implement envoyTLSVersions and envoyTLSCipherSuites * xds: merge TLS config * xds: configure TLS parameters with ingress TLS context from leaf * xds: nil check in resolveListenerTLSConfig validation * xds: nil check in makeTLSParameters* functions * changelog: add entry for TLS params on ingress config entries * xds: remove indirection for TLS params in TLSConfig structs * xds: return tlsContext, nil instead of ambiguous err Co-authored-by: Chris S. Kim <ckim@hashicorp.com> * xds: switch zero checks to types.TLSVersionUnspecified * ingress: add validation for ingress config entry TLS params * ingress: validate listener TLS config * xds: add basic ingress with TLS params tests * xds: add ingress listeners mixed TLS min version defaults precedence test * xds: add more explicit tests for ingress listeners inheriting gateway defaults * xds: add test for single TLS listener on gateway without TLS defaults * xds: regen golden files for TLSVersionInvalid zero value, add TLSVersionAuto listener test * types/tls: change TLSVersion to string * types/tls: update TLSCipherSuite to string type * types/tls: implement validation functions for TLSVersion and TLSCipherSuites, make some maps private * api: add TLS params to GatewayTLSConfig, add tests * api: add TLSMinVersion to ingress gateway config entry test JSON * xds: switch to Envoy TLS cipher suite encoding from types package * xds: fixup validation for TLSv1_3 min version with cipher suites * add some kitchen sink tests and add a missing struct tag * xds: check if mergedCfg.TLSVersion is in TLSVersionsWithConfigurableCipherSuites * xds: update connectTLSEnabled comment * xds: remove unsued resolveGatewayServiceTLSConfig function * xds: add makeCommonTLSContextFromLeafWithoutParams * types/tls: add LessThan comparator function for concrete values * types/tls: change tlsVersions validation map from string to TLSVersion keys * types/tls: remove unused envoyTLSCipherSuites * types/tls: enable chacha20 cipher suites for Consul agent * types/tls: remove insecure cipher suites from allowed config TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 and TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 are both explicitly listed as insecure and disabled in the Go source. Refs https://cs.opensource.google/go/go/+/refs/tags/go1.17.3:src/crypto/tls/cipher_suites.go;l=329-330 * types/tls: add ValidateConsulAgentCipherSuites function, make direct lookup map private * types/tls: return all unmatched cipher suites in validation errors * xds: check that Envoy API value matching TLS version is found when building TlsParameters * types/tls: check that value is found in map before appending to slice in MarshalEnvoyTLSCipherSuiteStrings * types/tls: cast to string rather than fmt.Printf in TLSCihperSuite.String() * xds: add TLSVersionUnspecified to list of configurable cipher suites * structs: update note about config entry warning * xds: remove TLS min version cipher suite unconfigurable test placeholder * types/tls: update tests to remove assumption about private map values Co-authored-by: R.B. Boyer <rb@hashicorp.com>	3 years ago
Dao Thanh Tung	88c7cfa578	URL-encode/decode resource names for HTTP API part 2 (#11957)	3 years ago
Daniel Nephin	d57dec5878	ca: remove unnecessary var, and slightly reduce cyclo complexity ... `newIntermediate` is always equal to `needsNewIntermediate`, so we can remove the extra variable and use the original directly. Also remove the `activeRoot.ID != newActiveRoot.ID` case from an if, because that case is already checked above, and `needsNewIntermediate` will already be true in that case. This condition now reads a lot better: > Persist a new root if we did not have one before, or if generated a new intermediate.	3 years ago
Daniel Nephin	0de7efb316	ca: remove unused provider.ActiveRoot call ... In the previous commit the single use of this storedRoot was removed. In this commit the original objective is completed. The Provider.ActiveRoot is being removed because 1. the secondary should get the active root from the Consul primary DC, not the provider, so that secondary DCs do not need to communicate with a provider instance in a different DC. 2. so that the Provider.ActiveRoot interface can be changed without impacting other code paths.	3 years ago
Daniel Nephin	d0578c6dfc	ca: extract the lookup of the active primary CA ... This method had only one caller, which always looked for the active root. This commit moves the lookup into the method to reduce the logic in the one caller. This is being done in preparation for a larger change. Keeping this separate so it is easier to see. The `storedRootID != primaryRoots.ActiveRootID` is being removed because these can never be different. The `storedRootID` comes from `provider.ActiveRoot`, the `primaryRoots.ActiveRootID` comes from the store `CARoot` from the primary. In both cases the source of the data is the primary DC. Technically they could be different if someone modified the provider outside of Consul, but that would break many things, so is not a supported flow. If these were out of sync because of ordering of events then the secondary will soon receive an update to `primaryRoots` and everything will be sorted out again.	3 years ago
Daniel Nephin	7121c78d34	ca: update godoc ... To clarify what to expect from the data stored in this field, and the behaviour of this function.	3 years ago
Daniel Nephin	abac8baa5d	ca: remove one call to provider.ActiveRoot ... ActiveRoot should not be called from the secondary DC, because there should not be a requirement to run the same Vault instance in a secondary DC. SignIntermediate is called in a secondary DC, so it should not call ActiveRoot We would also like to change the interface of ActiveRoot so that we can support using an intermediate cert as the primary CA in Consul. In preparation for making that change I am reducing the number of calls to ActiveRoot, so that there are fewer code paths to modify when the interface changes. This change required a change to the mockCAServerDelegate we use in tests. It was returning the RootCert for SignIntermediate, but that is not an accurate fake of production. In production this would also be a separate cert.	3 years ago
Daniel Nephin	eaa084fd41	ca: remove redundant append of an intermediate cert ... Immediately above this line we are already appending the full list of intermediates. The `provider.ActiveIntermediate` MUST be in this list of intermediates because it must be available to all the other non-leader Servers. If it was not in this list of intermediates then any proxy that received data from a non-leader would have the wrong certs. This is being removed now because we are planning on changing the `Provider.ActiveIntermediate` interface, and removing these extra calls ahead of time helps make that change easier.	3 years ago
Daniel Nephin	11f4cdaa49	ca: only generate a single private key for the whole test case ... Using tracing and cpu profiling I found that the majority of the time in these test cases is spent generating a private key. We really don't need separate private keys, so we can generate only one and use it for all cases. With this change the test runs much faster.	3 years ago
Daniel Nephin	b3ffe7ac72	ca: cleanup a test ... Fix the name to match the function it is testing Remove unused code Fix the signature, instead of returning (error, string) which should be (string, error) accept a testing.T to emit errors. Handle the error from encode.	3 years ago
Daniel Nephin	1fd6b16399	ca: use the new leaf signing lookup func in leader metrics	3 years ago
Blake Covarrubias	4bd92921f4	api: Return 404 when deregistering a non-existent check (#11950) ... Update the `/agent/check/deregister/` API endpoint to return a 404 HTTP response code when an attempt is made to de-register a check ID that does not exist on the agent. This brings the behavior of /agent/check/deregister/ in line with the behavior of /agent/service/deregister/ which was changed in #10632 to similarly return a 404 when de-registering non-existent services. Fixes #5821	3 years ago
Dhia Ayachi	1eac39ae9c	clone the service under lock to avoid a data race (#11940) ... * clone the service under lock to avoid a data race * add change log * create a struct and copy the pointer to mutate it to avoid a data race * fix failing test * revert added space * add comments, to clarify the data race.	3 years ago
Daniel Nephin	065f6f89fb	Merge pull request #11918 from hashicorp/dnephin/tob-followup ... Fix a few small bugs	3 years ago
Daniel Nephin	abfc1e4840	snapshot: return the error from replyFn ... The only function passed to SnapshotRPC today always returns a nil error, so there's no way to exercise this bug in practice. This change is being made for correctness so that it doesn't become a problem in the future, if we ever pass a different function to SnapshotRPC.	3 years ago
Daniel Nephin	0166b0839c	config: correctly capture all errors. ... Some calls to multierror.Append were not using the existing b.err, which meant we were losing all previous errors.	3 years ago
Chris S. Kim	4cd2542a3e	Fix test for ENT (#11946)	3 years ago
Chris S. Kim	e4bcaac08c	Fix test for ENT (#11941)	3 years ago
Dhia Ayachi	e653f81919	reset `coalesceTimer` to nil as soon as the event is consumed (#11924) ... * reset `coalesceTimer` to nil as soon as the event is consumed * add change log * refactor to add relevant test. * fix linter * Apply suggestions from code review Co-authored-by: Freddy <freddygv@users.noreply.github.com> * remove non needed check Co-authored-by: Freddy <freddygv@users.noreply.github.com>	3 years ago
Mathew Estafanous	0fdd1318e9	Ensure consistency with error-handling across all handlers. (#11599)	3 years ago
Jared Kirschner	b393c90ce7	Clarify service and check error messages (use ID) ... Error messages related to service and check operations previously included the following substrings: - service %q - check %q From this error message, it isn't clear that the expected field is the ID for the entity, not the name. For example, if the user has a service named test, the error message would read 'Unknown service "test"'. This is misleading - a service with that *name* does exist, but not with that *ID*. The substrings above have been modified to make it clear that ID is needed, not name: - service with ID %q - check with ID %q	3 years ago