239 Commits (9f58504f1c4e0dce5e43c72da6ae6dafa09f2849)

Author	SHA1	Message	Date
R.B. Boyer	c395affc93	connect: expose an API endpoint to compile the discovery chain (#6248) ... In addition to exposing compilation over the API cleaned up the structures that would be exchanged to be cleaner and easier to support and understand. Also removed ability to configure the envoy OverprovisioningFactor.	5 years ago
R.B. Boyer	dcb609af83	connect: detect and prevent circular discovery chain references (#6246)	5 years ago
R.B. Boyer	f02924fafe	connect: simplify the compiled discovery chain data structures (#6242) ... This should make them better for sending over RPC or the API. Instead of a chain implemented explicitly like a linked list (nodes holding pointers to other nodes) instead switch to a flat map of named nodes with nodes linking other other nodes by name. The shipped structure is just a map and a string to indicate which key to start from. Other changes: * inline the compiler option InferDefaults as true * introduce compiled target config to avoid needing to send back additional maps of Resolvers; future target-specific compiled state can go here * move compiled MeshGateway out of the Resolver and into the TargetConfig where it makes more sense.	5 years ago
R.B. Boyer	6393edba53	connect: reconcile how upstream configuration works with discovery chains (#6225) ... * connect: reconcile how upstream configuration works with discovery chains The following upstream config fields for connect sidecars sanely integrate into discovery chain resolution: - Destination Namespace/Datacenter: Compilation occurs locally but using different default values for namespaces and datacenters. The xDS clusters that are created are named as they normally would be. - Mesh Gateway Mode (single upstream): If set this value overrides any value computed for any resolver for the entire discovery chain. The xDS clusters that are created may be named differently (see below). - Mesh Gateway Mode (whole sidecar): If set this value overrides any value computed for any resolver for the entire discovery chain. If this is specifically overridden for a single upstream this value is ignored in that case. The xDS clusters that are created may be named differently (see below). - Protocol (in opaque config): If set this value overrides the value computed when evaluating the entire discovery chain. If the normal chain would be TCP or if this override is set to TCP then the result is that we explicitly disable L7 Routing and Splitting. The xDS clusters that are created may be named differently (see below). - Connect Timeout (in opaque config): If set this value overrides the value for any resolver in the entire discovery chain. The xDS clusters that are created may be named differently (see below). If any of the above overrides affect the actual result of compiling the discovery chain (i.e. "tcp" becomes "grpc" instead of being a no-op override to "tcp") then the relevant parameters are hashed and provided to the xDS layer as a prefix for use in naming the Clusters. This is to ensure that if one Upstream discovery chain has no overrides and tangentially needs a cluster named "api.default.XXX", and another Upstream does have overrides for "api.default.XXX" that they won't cross-pollinate against the operator's wishes. Fixes #6159	5 years ago
R.B. Boyer	8564b6bb38	connect: validate upstreams and prevent duplicates (#6224) ... * connect: validate upstreams and prevent duplicates * Actually run Upstream.Validate() instead of ignoring it as dead code. * Prevent two upstreams from declaring the same bind address and port. It wouldn't work anyway. * Prevent two upstreams from being declared that use the same type+name+namespace+datacenter. Due to how the Upstream.Identity() function worked this ended up mostly being enforced in xDS at use-time, but it should be enforced more clearly at register-time.	5 years ago
Paul Banks	e87cef2bb8	Revert "connect: support AWS PCA as a CA provider" (#6251) ... This reverts commit 3497b7c00d.	5 years ago
Todd Radel	3497b7c00d	connect: support AWS PCA as a CA provider (#6189) ... Port AWS PCA provider from consul-ent	5 years ago
Todd Radel	2552f4a11a	connect: Support RSA keys in addition to ECDSA (#6055) ... Support RSA keys in addition to ECDSA	5 years ago
R.B. Boyer	c6c4a2251a	Merge Consul OSS branch master at commit b3541c4f34	5 years ago
Jeff Mitchell	94c73d0c92	Chunking support (#6172) ... * Initial chunk support This uses the go-raft-middleware library to allow for chunked commits to the KV	5 years ago
Matt Keeler	3053342198	Envoy Mesh Gateway integration tests (#6187) ... * Allow setting the mesh gateway mode for an upstream in config files * Add envoy integration test for mesh gateways This necessitated many supporting changes in most of the other test cases. Add remote mode mesh gateways integration test	5 years ago
R.B. Boyer	ad9e7b6ae9	connect: allow L7 routers to match on http methods (#6164) ... Fixes #6158	5 years ago
R.B. Boyer	85cf2706e6	connect: change router syntax for matching query parameters to resemble the syntax for matching paths and headers for consistency. (#6163) ... This is a breaking change, but only in the context of the beta series.	5 years ago
R.B. Boyer	1dbd92e091	connect: validate and test more of the L7 config entries (#6156)	5 years ago
R.B. Boyer	e039dfd7f8	connect: rework how the service resolver subset OnlyPassing flag works (#6173) ... The main change is that we no longer filter service instances by health, preferring instead to render all results down into EDS endpoints in envoy and merely label the endpoints as HEALTHY or UNHEALTHY. When OnlyPassing is set to true we will force consul checks in a 'warning' state to render as UNHEALTHY in envoy. Fixes #6171	5 years ago
Matt Keeler	d7fe8befa9	Update go-bexpr (#6190) ... * Update go-bexpr to v0.1.1 This brings in: • `in`/`not in` operators to do substring matching • `matches` / `not matches` operators to perform regex string matching. * Add the capability to auto-generate the filtering selector ops tables for our docs	5 years ago
Matt Keeler	4728329aeb	Various Gateway Fixes (#6093) ... * Ensure the mesh gateway configuration comes back in the api within each upstream * Add a test for the MeshGatewayConfig in the ToAPI functions * Ensure we don’t use gateways for dc local connections * Update the svc kind index for deletions * Replace the proxycfg.state cache with an interface for testing Also start implementing proxycfg state testing. * Update the state tests to verify some gateway watches for upstream-targets of a discovery chain.	5 years ago
R.B. Boyer	bcd2de3a2e	implement some missing service-router features and add more xDS testing (#6065) ... - also implement OnlyPassing filters for non-gateway clusters	5 years ago
R.B. Boyer	9138a97054	Fix bug in service-resolver redirects if the destination uses a default resolver. (#6122) ... Also: - add back an internal http endpoint to dump a compiled discovery chain for debugging purposes Before the CompiledDiscoveryChain.IsDefault() method would test: - is this chain just one resolver step? - is that resolver step just the default? But what I forgot to test: - is that resolver step for the same service that the chain represents? This last point is important because if you configured just one config entry: kind = "service-resolver" name = "web" redirect { service = "other" } and requested the chain for "web" you'd get back a **default** resolver for "other". In the xDS code the IsDefault() method is used to determine if this chain is "empty". If it is then we use the pre-discovery-chain logic that just uses data embedded in the Upstream object (and still lets the escape hatches function). In the example above that means certain parts of the xDS code were going to try referencing a cluster named "web..." despite the other parts of the xDS code maintaining clusters named "other...".	5 years ago
R.B. Boyer	67a36e3452	handle structs.ConfigEntry decoding similarly to api.ConfigEntry decoding (#6106) ... Both 'consul config write' and server bootstrap config entries take a decoding detour through mapstructure on the way from HCL to an actual struct. They both may take in snake_case or CamelCase (for consistency) so need very similar handling. Unfortunately since they are operating on mirror universes of structs (api.* vs structs.*) the code cannot be identitical, so try to share the kind-configuration and duplicate the rest for now.	5 years ago
Matt Keeler	6e65811db2	Envoy CLI bind addresses (#6107) ... * Ensure we MapWalk the proxy config in the NodeService and ServiceNode structs This gets rid of some json encoder errors in the catalog endpoints * Allow passing explicit bind addresses to envoy * Move map walking to the ConnectProxyConfig struct Any place where this struct gets JSON encoded will benefit as opposed to having to implement it everywhere. * Fail when a non-empty address is provided and not bindable * camel case * Update command/connect/envoy/envoy.go Co-Authored-By: Paul Banks <banks@banksco.de>	5 years ago
Matt Keeler	3eb3ee5a15	Merge pull request #6053 from hashicorp/gateways_and_resolvers ... Integrate Mesh Gateways with ServiceResolverSubsets	5 years ago
R.B. Boyer	43770b9391	digest the proxy-defaults protocol into the graph (#6050)	5 years ago
Matt Keeler	3b6d5e382a	Implement caching for config entry lists ... Update agent/cache-types/config_entry.go Co-Authored-By: R.B. Boyer <public@richardboyer.net>	5 years ago
R.B. Boyer	4bdb690a25	activate most discovery chain features in xDS for envoy (#6024)	5 years ago
Matt Keeler	bdebe62fd0	Fix some tests that I broke when refactoring the ConfigSnapshot (#6051) ... * Fix some tests that I broke when refactoring the ConfigSnapshot * Make sure the MeshGateway config is added to all the right api structs * Fix some more tests	5 years ago
Matt Keeler	8d953f5840	Implement Mesh Gateways ... This includes both ingress and egress functionality.	5 years ago
Matt Keeler	4bc1277315	Include a content hash of the intention for use during replication	5 years ago
Matt Keeler	3943e38133	Implement Kind based ServiceDump and caching of the ServiceDump RPC	5 years ago
R.B. Boyer	2ad516aeaf	do some initial config entry graph validation during writes (#6047)	5 years ago
hashicorp-ci	43bda6fb76	Merge Consul OSS branch 'master' at commit e91f73f592	5 years ago
Hans Hasselberg	33a7df3330	tls: auto_encrypt enables automatic RPC cert provisioning for consul clients (#5597)	5 years ago
R.B. Boyer	6a52f9f9fb	initial version of L7 config entry compiler (#5994) ... With this you should be able to fetch all of the relevant discovery chain config entries from the state store in one query and then feed them into the compiler outside of a transaction. There are a lot of TODOs scattered through here, but they're mostly around handling fun edge cases and can be deferred until more of the plumbing works completely.	5 years ago
R.B. Boyer	ceef44bbc9	adding new config entries for L7 discovery chain (unused) (#5987)	6 years ago
hashicorp-ci	f4304e2e5b	Merge Consul OSS branch 'master' at commit 4eb73973b6	6 years ago
Pierre Souchay	0e907f5aa8	Support for maximum size for Output of checks (#5233) ... * Support for maximum size for Output of checks This PR allows users to limit the size of output produced by checks at the agent and check level. When set at the agent level, it will limit the output for all checks monitored by the agent. When set at the check level, it can override the agent max for a specific check but only if it is lower than the agent max. Default value is 4k, and input must be at least 1.	6 years ago
Matt Keeler	43c5ba0304	New Cache Types (#5995) ... * Add a cache type for the Catalog.ListServices endpoint * Add a cache type for the Catalog.ListDatacenters endpoint	6 years ago
Aestek	b839f52195	kv: do not trigger watches when setting the same value (#5885) ... If a KVSet is performed but does not update the entry, do not trigger watches for this key. This avoids releasing blocking queries for KV values that did not actually changed.	6 years ago
Matt Keeler	f3d9b999ee	Add tagged addresses for services (#5965) ... This allows addresses to be tagged at the service level similar to what we allow for nodes already. The address translation that can be enabled with the `translate_wan_addrs` config was updated to take these new addresses into account as well.	6 years ago
R.B. Boyer	40336fd353	agent: fix several data races and bugs related to node-local alias checks (#5876) ... The observed bug was that a full restart of a consul datacenter (servers and clients) in conjunction with a restart of a connect-flavored application with bring-your-own-service-registration logic would very frequently cause the envoy sidecar service check to never reflect the aliased service. Over the course of investigation several bugs and unfortunate interactions were corrected: (1) local.CheckState objects were only shallow copied, but the key piece of data that gets read and updated is one of the things not copied (the underlying Check with a Status field). When the stock code was run with the race detector enabled this highly-relevant-to-the-test-scenario field was found to be racy. Changes: a) update the existing Clone method to include the Check field b) copy-on-write when those fields need to change rather than incrementally updating them in place. This made the observed behavior occur slightly less often. (2) If anything about how the runLocal method for node-local alias check logic was ever flawed, there was no fallback option. Those checks are purely edge-triggered and failure to properly notice a single edge transition would leave the alias check incorrect until the next flap of the aliased check. The change was to introduce a fallback timer to act as a control loop to double check the alias check matches the aliased check every minute (borrowing the duration from the non-local alias check logic body). This made the observed behavior eventually go away when it did occur. (3) Originally I thought there were two main actions involved in the data race: A. The act of adding the original check (from disk recovery) and its first health evaluation. B. The act of the HTTP API requests coming in and resetting the local state when re-registering the same services and checks. It took awhile for me to realize that there's a third action at work: C. The goroutines associated with the original check and the later checks. The actual sequence of actions that was causing the bad behavior was that the API actions result in the original check to be removed and re-added _without waiting for the original goroutine to terminate_. This means for brief windows of time during check definition edits there are two goroutines that can be sending updates for the alias check status. In extremely unlikely scenarios the original goroutine sees the aliased check start up in `critical` before being removed but does not get the notification about the nearly immediate update of that check to `passing`. This is interlaced wit the new goroutine coming up, initializing its base case to `passing` from the current state and then listening for new notifications of edge triggers. If the original goroutine "finishes" its update, it then commits one more write into the local state of `critical` and exits leaving the alias check no longer reflecting the underlying check. The correction here is to enforce that the old goroutines must terminate before spawning the new one for alias checks.	6 years ago
R.B. Boyer	20eefeea11	acl: a role binding rule for a role that does not exist should be ignored (#5778) ... I wrote the docs under this assumption but completely forgot to actually enforce it.	6 years ago
R.B. Boyer	b4371bcccd	acl: enforce that you cannot persist tokens and roles with missing links except during replication (#5779)	6 years ago
Matt Keeler	42d32db817	Fix ConfigEntryResponse binary marshaller and ensure we watch the chan in ConfigEntry.Get even when no entry exists. (#5773)	6 years ago
Paul Banks	8f5b16ebaf	Fix uint8 conversion issues for service config response maps.	6 years ago
Paul Banks	0cfb6051ea	Add integration test for central config; fix central config WIP (#5752) ... * Add integration test for central config; fix central config WIP * Add integration test for central config; fix central config WIP * Set proxy protocol correctly and begin adding upstream support * Add upstreams to service config cache key and start new notify watcher if they change. This doesn't update the tests to pass though. * Fix some merging logic get things working manually with a hack (TODO fix properly) * Simplification to not allow enabling sidecars centrally - it makes no sense without upstreams anyway * Test compile again and obvious ones pass. Lots of failures locally not debugged yet but may be flakes. Pushing up to see what CI does * Fix up service manageer and API test failures * Remove the enable command since it no longer makes much sense without being able to turn on sidecar proxies centrally * Remove version.go hack - will make integration test fail until release * Remove unused code from commands and upstream merge * Re-bump version to 1.5.0	6 years ago
Matt Keeler	69f902608c	Update to use a consulent build tag instead of just ent (#5759)	6 years ago
Matt Keeler	d0f410cd84	Make a few config entry endpoints return 404s and allow for snake_case and lowercase key names. (#5748)	6 years ago
Matt Keeler	4daa1585b0	ACL Token ID Initialization (#5307)	6 years ago
Kyle Havlovitz	aba54cec55	Add HTTP endpoints for config entry management (#5718)	6 years ago
Paul Banks	421ecd32fc	Connect: allow configuring Envoy for L7 Observability (#5558) ... * Add support for HTTP proxy listeners * Add customizable bootstrap configuration options * Debug logging for xDS AuthZ * Add Envoy Integration test suite with basic test coverage * Add envoy command tests to cover new cases * Add tracing integration test * Add gRPC support WIP * Merged changes from master Docker. get CI integration to work with same Dockerfile now * Make docker build optional for integration * Enable integration tests again! * http2 and grpc integration tests and fixes * Fix up command config tests * Store all container logs as artifacts in circle on fail * Add retries to outer part of stats measurements as we keep missing them in CI * Only dump logs on failing cases * Fix typos from code review * Review tidying and make tests pass again * Add debug logs to exec test. * Fix legit test failure caused by upstream rename in envoy config * Attempt to reduce cases of bad TLS handshake in CI integration tests * bring up the right service * Add prometheus integration test * Add test for denied AuthZ both HTTP and TCP * Try ANSI term for Circle	6 years ago