consul

Commit Graph

Author	SHA1	Message	Date
Daniel Nephin	e8312d6b5a	testing: remove unnecessary calls to freeport Previously we believe it was necessary for all code that required ports to use freeport to prevent conflicts. https://github.com/dnephin/freeport-test shows that it is actually save to use port 0 (`127.0.0.1:0`) as long as it is passed directly to `net.Listen`, and the listener holds the port for as long as it is needed. This works because freeport explicitly avoids the ephemeral port range, and port 0 always uses that range. As you can see from the test output of https://github.com/dnephin/freeport-test, the two systems never use overlapping ports. This commit converts all uses of freeport that were being passed directly to a net.Listen to use port 0 instead. This allows us to remove a bit of wrapping we had around httptest, in a couple places.	3 years ago
Daniel Nephin	d795a73f78	testing: use the new freeport interfaces	3 years ago
Daniel Nephin	8c7475d95e	sdk: add freeport functions that use t.Cleanup	3 years ago
Mark Anderson	7e8228a20b	Remove some usage of md5 from the system (#11491 ) * Remove some usage of md5 from the system OSS side of https://github.com/hashicorp/consul-enterprise/pull/1253 This is a potential security issue because an attacker could conceivably manipulate inputs to cause persistence files to collide, effectively deleting the persistence file for one of the colliding elements. Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Daniel Nephin	51d8417545	Merge pull request #10690 from tarat44/h2c-support-in-ping-checks add support for h2c in h2 ping health checks	3 years ago
Dhia Ayachi	2801785710	regenerate expired certs (#11462 ) * regenerate expired certs * add documentation to generate tests certificates	3 years ago
R.B. Boyer	ef559dfdd4	agent: refactor the agent delegate interface to be partition friendly (#11429 )	3 years ago
tarat44	1e8e44d442	fix formatting	3 years ago
tarat44	c1ed3a9a94	change config option to H2PingUseTLS	3 years ago
tarat44	3c9f5a73d9	add support for h2c in h2 ping health checks	3 years ago
R.B. Boyer	5b6d96d27d	grpc: ensure that streaming gRPC requests work over mesh gateway based wan federation (#10838 ) Fixes #10796	3 years ago
Evan Culver	710bd90ef7	checks: Add Interval and Timeout to API response (#10717 )	3 years ago
R.B. Boyer	fc9b1a277d	sync changes to oss files made in enterprise (#10670 )	3 years ago
R.B. Boyer	188e8dc51f	agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning (#10669 )	3 years ago
Daniel Nephin	970f5d78ec	agent: fix two data race in agent tests The LogOutput io.Writer used by TestAgent must allow concurrent reads and writes, and a bytes.Buffer does not allow this. The bytes.Buffer must be wrapped with a lock to make this safe.	3 years ago
Daniel Nephin	3a045cca8d	ca: remove unused RotationPeriod field This field was never used. Since it is persisted as part of a map[string]interface{} it is pretty easy to remove it.	3 years ago
Daniel Nephin	8b365f8271	Remove a racy and failing test This test is super racy (it's not just a single line). This test also starts failing once streaming is enabled, because the cache rate limit no longer applies to the requests in the test. The queries use streaming instead of the cache. This test is no longer valid, and the functionality is already well tested by TestCacheThrottle. Instead of spending time rewriting this test, let's remove it. ``` WARNING: DATA RACE Read at 0x00c01de410fc by goroutine 735: github.com/hashicorp/consul/agent.TestCacheRateLimit.func1() /home/daniel/pers/code/consul/agent/agent_test.go:1024 +0x9af github.com/hashicorp/consul/testrpc.WaitForTestAgent() /home/daniel/pers/code/consul/testrpc/wait.go:99 +0x209 github.com/hashicorp/consul/agent.TestCacheRateLimit.func1() /home/daniel/pers/code/consul/agent/agent_test.go:966 +0x1ad testing.tRunner() /usr/lib/go/src/testing/testing.go:1193 +0x202 Previous write at 0x00c01de410fc by goroutine 605: github.com/hashicorp/consul/agent.TestCacheRateLimit.func1.2() /home/daniel/pers/code/consul/agent/agent_test.go:998 +0xe9 Goroutine 735 (running) created at: testing.(*T).Run() /usr/lib/go/src/testing/testing.go:1238 +0x5d7 github.com/hashicorp/consul/agent.TestCacheRateLimit() /home/daniel/pers/code/consul/agent/agent_test.go:961 +0x375 testing.tRunner() /usr/lib/go/src/testing/testing.go:1193 +0x202 Goroutine 605 (finished) created at: github.com/hashicorp/consul/agent.TestCacheRateLimit.func1() /home/daniel/pers/code/consul/agent/agent_test.go:1022 +0x91e github.com/hashicorp/consul/testrpc.WaitForTestAgent() /home/daniel/pers/code/consul/testrpc/wait.go:99 +0x209 github.com/hashicorp/consul/agent.TestCacheRateLimit.func1() /home/daniel/pers/code/consul/agent/agent_test.go:966 +0x1ad testing.tRunner() /usr/lib/go/src/testing/testing.go:1193 +0x202 ```	3 years ago
R.B. Boyer	a2876453a5	connect/ca: cease including the common name field in generated certs (#10424 ) As part of this change, we ensure that the SAN extensions are marked as critical when the subject is empty so that AWS PCA tolerates the loss of common names well and continues to function as a Connect CA provider. Parts of this currently hack around a bug in crypto/x509 and can be removed after https://go-review.googlesource.com/c/go/+/329129 lands in a Go release. Note: the AWS PCA tests do not run automatically, but the following passed locally for me: ENABLE_AWS_PCA_TESTS=1 go test ./agent/connect/ca -run TestAWS	3 years ago
Matt Keeler	234d0a3c2a	Preparation for changing where license management is done.	4 years ago
Iryna Shustava	d7d44f6ae7	Save exposed ports in agent's store and expose them via API (#10173 ) * Save exposed HTTP or GRPC ports to the agent's store * Add those the health checks API so we can retrieve them from the API * Change redirect-traffic command to also exclude those ports from inbound traffic redirection when expose.checks is set to true.	4 years ago
Daniel Nephin	e229b877d8	health: create health.Client in Agent.New	4 years ago
Tara Tufano	9deb52e868	add http2 ping health checks (#8431 ) * add http2 ping checks * fix test issue * add h2ping check to config resources * add new test and docs for h2ping * fix grammatical inconsistency in H2PING documentation * resolve rebase conflicts, add test for h2ping tls verification failure * api documentation for h2ping * update test config data with H2PING * add H2PING to protocol buffers and update changelog * fix typo in changelog entry	4 years ago
Daniel Nephin	f36d77ee4a	Merge pull request #9301 from hashicorp/dnephin/add-service-2 agent: reduce AddService 2	4 years ago
R.B. Boyer	c608dc0d60	server: initialize mgw-wanfed to use local gateways more on startup (#9528 ) Fixes #9342	4 years ago
Daniel Nephin	1495291054	agent: move two fields off of AddServiceRequest	4 years ago
Daniel Nephin	3d39359bcb	agent: move deprecated AddServiceFromSource to a test file The method is only used in tests, and only exists for legacy calls. There was one other package which used this method in tests. Export the AddServiceRequest and a couple of its fields so the new function can be used in those tests.	4 years ago
Daniel Nephin	6757231b82	agent: rename AddService->AddServiceFromSource In preparation for extracting a single AddService func that accepts a request struct.	4 years ago
Daniel Nephin	d113f0e690	structs: Fix printing of IDs These types are used as values (not pointers) in other structs. Using a pointer receiver causes problems when the value is printed. fmt will not call the String method if it is passed a value and the String method has a pointer receiver. By using a value receiver the correct string is printed. Also remove some unused methods.	4 years ago
Michael Montgomery	e4f603dfae	Merge branch 'master' into 6074-allow-config-MaxHeaderBytes	4 years ago
Michael Montgomery	817903b925	Fixed failing tests Removed use of `NewTestAgent`, per review comment Removed CLI flag, per review comment Updated website documentation Added changelog entry	4 years ago
Daniel Nephin	b9e60c0775	testing: skip slow tests with -short Add a skip condition to all tests slower than 100ms. This change was made using `gotestsum tool slowest` with data from the last 3 CI runs of master. See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests With this change: ``` $ time go test -count=1 -short ./agent ok github.com/hashicorp/consul/agent 0.743s real 0m4.791s $ time go test -count=1 -short ./agent/consul ok github.com/hashicorp/consul/agent/consul 4.229s real 0m8.769s ```	4 years ago
Michael Montgomery	585c84e9ff	Merge branch 'master' into 6074-allow-config-MaxHeaderBytes	4 years ago
Daniel Nephin	79963be559	Use freeport To prevent other tests which already use freeport from flaking when port 0 steals their reserved port.	4 years ago
Daniel Nephin	738bf9efdc	agent: fix bug with multiple listeners Previously the listener was being passed to a closure in a loop without capturing the loop variable. The result is only the last listener is used, so the http/https servers only listen on one address. This problem is fixed by capturing the variable by passing it into a function.	4 years ago
Michael Montgomery	5b6ac035ff	Resolves #6074 . Adds new option to configure HTTP Server's MaxHeaderBytes with option `-http-max-header-bytes` Adds tests for behavior	4 years ago
Mathilde Gilles	1c8369b3c3	Fix: service LocallyRegisteredAsSidecar property is not persisted When a service is deregistered, we check whever matching services were registered as sidecar along with it and deregister them as well. To determine if a service is indeed a sidecar we check the structs.ServiceNode.LocallyRegisteredAsSidecar property. However, to avoid interal API leakage, it is excluded from JSON serialization, meaning it is not persisted to disk either. When the agent is restarted, this property lost and sidecars are no longer deregistered along with their parent service. To fix this, we now specifically save this property in the persisted service file.	4 years ago
R.B. Boyer	9801ef8eb1	agent: enable enable_central_service_config by default (#8746 )	4 years ago
R.B. Boyer	7eef25daf5	agent: when enable_central_service_config is enabled ensure agent reload doesn't revert check state to critical (#8747 ) Likely introduced when #7345 landed.	4 years ago
Daniel Nephin	c18516ad7d	Merge pull request #8680 from hashicorp/dnephin/replace-consul-opts-with-base-deps agent: Repalce ConsulOptions with a new struct from agent.BaseDeps	4 years ago
Daniel Nephin	cdd392d77f	agent/consul: pass dependencies directly from agent In an upcoming change we will need to pass a grpc.ClientConnPool from BaseDeps into Server. While looking at that change I noticed all of the existing consulOption fields are already on BaseDeps. Instead of duplicating the fields, we can create a struct used by agent/consul, and use that struct in BaseDeps. This allows us to pass along dependencies without translating them into different representations. I also looked at moving all of BaseDeps in agent/consul, however that created some circular imports. Resolving those cycles wouldn't be too bad (it was only an error in agent/consul being imported from cache-types), however this change seems a little better by starting to introduce some structure to BaseDeps. This change is also a small step in reducing the scope of Agent. Also remove some constants that were only used by tests, and move the relevant comment to where the live configuration is set. Removed some validation from NewServer and NewClient, as these are not really runtime errors. They would be code errors, which will cause a panic anyway, so no reason to handle them specially here.	4 years ago
Daniel Nephin	0bb9c318b7	http: fix tests incorrectly using HTTPAddr to get the address of the https server. In #8234 I changed a few tests to use TestAgent.HTTPAddr() to find the addr used in the test. Due to the way HTTPAddr() was implemented these tests were passing, but I think the pass was incidental. HTTPAddr() was not matching any servers, and was instead returning the last server, which happened to be the one these tests wanted. This commit fixes the implementation of HTTPAddr to panic if no match was found. The tests which require an HTTPS server are changed to use a new firstAddr() to look up the correct address.	4 years ago
Daniel Nephin	4c9ed41eab	Merge pull request #8554 from hashicorp/dnephin/agent-setup-persisted-tokens agent: move token persistence from agent into token.Store	4 years ago
Daniel Nephin	6ca45e1a61	agent: add apiServers type for managing HTTP servers Remove Server field from HTTPServer. The field is no longer used.	4 years ago
Daniel Nephin	330be5b740	agent/token: Move token persistence out of agent And into token.Store. This change isolates any awareness of token persistence in a single place. It is a small step in allowing Agent.New to accept its dependencies.	4 years ago
Daniel Nephin	72bf350069	Merge pull request #8552 from pierresouchay/reload_cache_throttling_config Ensure that Cache options are reloaded when `consul reload` is performed	4 years ago
Pierre Souchay	9a64d3e5fe	Also test reload of EntryFetchMaxBurst	4 years ago
Matt Keeler	f97cc0445a	Move RPC router from Client/Server and into BaseDeps (#8559 ) This will allow it to be a shared component which is needed for AutoConfig	4 years ago
Pierre Souchay	5842a902df	Tests that changes in rate limit are taken into account by agent	4 years ago
Daniel Nephin	d68edcecf4	testing: Remove all the defer os.Removeall Now that testutil uses t.Cleanup to remove the directory the caller no longer has to manage the removal	4 years ago
Daniel Nephin	3a4e62836b	testing: Remove TestAgent.Key and change TestAgent.DataDir TestAgent.Key was only used by 3 tests. Extracting it from the common helper that is used in hundreds of tests helps keep the shared part small and more focused. This required a second change (which I was planning on making anyway), which was to change the behaviour of DataDir. Now in all cases the TestAgent will use the DataDir, and clean it up once the test is complete.	4 years ago
Daniel Nephin	0738eb8596	Extract nodeID functions to a different file In preparation for turning them into functions. To reduce the scope of Agent, and refactor how Agent is created and started.	4 years ago
Daniel Nephin	38980ebb4c	config: Make Source an interface This will allow us to accept config from auto-config without needing to go through a serialziation cycle.	4 years ago
Matt Keeler	1a78cf9b4c	Ensure certificates retrieved through the cache get persisted with auto-config (#8409 )	4 years ago
Matt Keeler	34034b76f5	Agent Auto Config: Implement Certificate Generation (#8360 ) Most of the groundwork was laid in previous PRs between adding the cert-monitor package to extracting the logic of signing certificates out of the connect_ca_endpoint.go code and into a method on the server. This also refactors the auto-config package a bit to split things out into multiple files.	4 years ago
Matt Keeler	83d09de230	Fix some broken code in master There were several PRs that while all passed CI independently, when they all got merged into the same branch caused compilation errors in test code. The main changes that caused issues where changing agent/cache.Cache.New to require a concrete options struct instead of a pointer. This broke the cert monitor tests and the catalog_list_services_test.go. Another change was made to unembed the http.Server from the agent.HTTPServer struct. That coupled with another change to add a test to ensure cache rate limiting coming from HTTP requests was working as expected caused compilation failures.	4 years ago
Pierre Souchay	505de6dc29	Added ratelimit to handle throtling cache (#8226 ) This implements a solution for #7863 It does: Add a new config cache.entry_fetch_rate to limit the number of calls/s for a given cache entry, default value = rate.Inf Add cache.entry_fetch_max_burst size of rate limit (default value = 2) The new configuration now supports the following syntax for instance to allow 1 query every 3s: command line HCL: -hcl 'cache = { entry_fetch_rate = 0.333}' in JSON { "cache": { "entry_fetch_rate": 0.333 } }	4 years ago
Matt Keeler	6e7acfa618	Add an AutoEncrypt “integration” test Also fix a bug where Consul could segfault if TLS was enabled but no client certificate was provided. How no one has reported this as a problem I am not sure.	4 years ago
Daniel Nephin	0279bf6fe5	Update TestAgent_GetCoordinate The old test case was a very specific regresion test for a case that is no longer possible. Replaced with a new test that checks the default coordinate is returned.	4 years ago
Daniel Nephin	010a609912	Fix a bunch of unparam lint issues	4 years ago
Matt Keeler	b0fcf86140	Change auto config authorizer to allow for future extension The envisioned changes would allow extra settings to enable dynamically defined auth methods to be used instead of or in addition to the statically defined one in the configuration.	5 years ago
Matt Keeler	3dbbd2d37d	Implement Client Agent Auto Config There are a couple of things in here. First, just like auto encrypt, any Cluster.AutoConfig RPC will implicitly use the less secure RPC mechanism. This drastically modifies how the Consul Agent starts up and moves most of the responsibilities (other than signal handling) from the cli command and into the Agent.	5 years ago
Daniel Nephin	068b43df90	Enable gofmt simplify Code changes done automatically with 'gofmt -s -w'	5 years ago
Hans Hasselberg	dfcf45c6cf	tests: use constructor instead init (#8024 )	5 years ago
Pierre Souchay	9813ae512b	checks: when a service does not exists in an alias, consider it failing (#7384 ) In current implementation of Consul, check alias cannot determine if a service exists or not. Because a service without any check is semantically considered as passing, so when no healthchecks are found for an agent, the check was considered as passing. But this make little sense as the current implementation does not make any difference between: * a non-existing service (passing) * a service without any check (passing as well) In order to make it work, we have to ensure that when a check did not find any healthcheck, the service does indeed exists. If it does not, lets consider the check as failing.	5 years ago
Daniel Nephin	99eb583ebc	Replace goe/verify.Values with testify/require.Equal (#7993 ) * testing: replace most goe/verify.Values with require.Equal One difference between these two comparisons is that go/verify considers nil slices/maps to be equal to empty slices/maps, where as testify/require does not, and does not appear to provide any way to enable that behaviour. Because of this difference some expected values were changed from empty slices to nil slices, and some calls to verify.Values were left. * Remove github.com/pascaldekloe/goe/verify Reduce the number of assertion packages we use from 2 to 1	5 years ago
Daniel Nephin	600645b5f9	Add unconvert linter To find unnecessary type convertions	5 years ago
Pierre Souchay	2a8bf45e38	agent: show warning when enable_script_checks is enabled without safty net (#7437 ) In order to enforce a bit security on Consul agents, add a new method in agent to highlight possible security issues. This does not return an error for now, but might in the future. For now, it detects issues such as: https://www.hashicorp.com/blog/protecting-consul-from-rce-risk-in-specific-configurations/ This would display this kind of messages: ``` 2020-03-11T18:27:49.873+0100 [ERROR] agent: [SECURITY] issue: error="using enable-script-checks without ACLs and without allow_write_http_from is DANGEROUS, use enable-local-script-checks instead see https://www.hashicorp.com/blog/protecting-consul-from-rce-risk-in-specific-configurations/" ```	5 years ago
Daniel Nephin	e759daafdd	Rename NewTestAgentWithFields to StartTestAgent This function now only starts the agent. Using: git grep -l 'StartTestAgent(t, true,' \| \ xargs sed -i -e 's/StartTestAgent(t, true,/StartTestAgent(t,/g'	5 years ago
Daniel Nephin	f9f6b14533	Convert the remaining calls to NewTestAgentWithFields After removing the t.Name() parameter with sed, convert the last few tests which use a custom name to call NewTestAgentWithFields instead.	5 years ago
Daniel Nephin	475659a132	Remove name from NewTestAgent Using: git grep -l 'NewTestAgent(t, t.Name(),' \| \ xargs sed -i -e 's/NewTestAgent(t, t.Name(),/NewTestAgent(t,/g'	5 years ago
Daniel Nephin	ad7c78f134	Remove t.Name() from TestAgent.Name And re-add the name to the logger so that log messages from different agents in a single can be identified.	5 years ago
Daniel Nephin	dd40a1535e	testing: reduce verbosity of output log Previously the log output included the test name twice and a long date format. The test output is already grouped by test, so adding the test name did not add any new information. The date and time are only useful to understand elapsed time, so using a short format should provide succident detail. Also fixed a bug in NewTestAgentWithFields where nil was returned instead of the test agent.	5 years ago
Chris Piraino	136099d834	Fix flakey health check reload test (#7490 ) This test would occasionally fail because we checked for a status of "critical" initially. This races with the actual healthcheck being run and declared passing. We instead use a ttl health check so that we don't rely on timing at all.	5 years ago
R.B. Boyer	6adad71125	wan federation via mesh gateways (#6884 ) This is like a Möbius strip of code due to the fact that low-level components (serf/memberlist) are connected to high-level components (the catalog and mesh-gateways) in a twisty maze of references which make it hard to dive into. With that in mind here's a high level summary of what you'll find in the patch: There are several distinct chunks of code that are affected: * new flags and config options for the server * retry join WAN is slightly different * retry join code is shared to discover primary mesh gateways from secondary datacenters * because retry join logic runs in the agent and the results of that operation for primary mesh gateways are needed in the server there are some methods like `RefreshPrimaryGatewayFallbackAddresses` that must occur at multiple layers of abstraction just to pass the data down to the right layer. * new cache type `FederationStateListMeshGatewaysName` for use in `proxycfg/xds` layers * the function signature for RPC dialing picked up a new required field (the node name of the destination) * several new RPCs for manipulating a FederationState object: `FederationState:{Apply,Get,List,ListMeshGateways}` * 3 read-only internal APIs for debugging use to invoke those RPCs from curl * raft and fsm changes to persist these FederationStates * replication for FederationStates as they are canonically stored in the Primary and replicated to the Secondaries. * a special derivative of anti-entropy that runs in secondaries to snapshot their local mesh gateway `CheckServiceNodes` and sync them into their upstream FederationState in the primary (this works in conjunction with the replication to distribute addresses for all mesh gateways in all DCs to all other DCs) * a "gateway locator" convenience object to make use of this data to choose the addresses of gateways to use for any given RPC or gossip operation to a remote DC. This gets data from the "retry join" logic in the agent and also directly calls into the FSM. * RPC (`:8300`) on the server sniffs the first byte of a new connection to determine if it's actually doing native TLS. If so it checks the ALPN header for protocol determination (just like how the existing system uses the type-byte marker). * 2 new kinds of protocols are exclusively decoded via this native TLS mechanism: one for ferrying "packet" operations (udp-like) from the gossip layer and one for "stream" operations (tcp-like). The packet operations re-use sockets (using length-prefixing) to cut down on TLS re-negotiation overhead. * the server instances specially wrap the `memberlist.NetTransport` when running with gateway federation enabled (in a `wanfed.Transport`). The general gist is that if it tries to dial a node in the SAME datacenter (deduced by looking at the suffix of the node name) there is no change. If dialing a DIFFERENT datacenter it is wrapped up in a TLS+ALPN blob and sent through some mesh gateways to eventually end up in a server's :8300 port. * a new flag when launching a mesh gateway via `consul connect envoy` to indicate that the servers are to be exposed. This sets a special service meta when registering the gateway into the catalog. * `proxycfg/xds` notice this metadata blob to activate additional watches for the FederationState objects as well as the location of all of the consul servers in that datacenter. * `xds:` if the extra metadata is in place additional clusters are defined in a DC to bulk sink all traffic to another DC's gateways. For the current datacenter we listen on a wildcard name (`server.<dc>.consul`) that load balances all servers as well as one mini-cluster per node (`<node>.server.<dc>.consul`) * the `consul tls cert create` command got a new flag (`-node`) to help create an additional SAN in certs that can be used with this flavor of federation.	5 years ago
Pierre Souchay	864f7efffa	agent: configuration reload preserves check's statuses for services (#7345 ) This fixes issue #7318 Between versions 1.5.2 and 1.5.3, a regression has been introduced regarding health of services. A patch #6144 had been issued for HealthChecks of nodes, but not for healthchecks of services. What happened when a reload was: 1. save all healthcheck statuses 2. cleanup everything 3. add new services with healthchecks In step 3, the state of healthchecks was taken into account locally, so at step 3, but since we cleaned up at step 2, state was lost. This PR introduces the snap parameter, so step 3 can use information from step 1	5 years ago
Chris Piraino	401221de58	Allow users to configure either unstructured or JSON logging (#7130 ) * hclog Allow users to choose between unstructured and JSON logging	5 years ago
Aestek	ba8fd8296f	Add support for dual stack IPv4/IPv6 network (#6640 ) * Use consts for well known tagged adress keys * Add ipv4 and ipv6 tagged addresses for node lan and wan * Add ipv4 and ipv6 tagged addresses for service lan and wan * Use IPv4 and IPv6 address in DNS	5 years ago
Matt Keeler	5934f803bf	Sync of OSS changes to support namespaces (#6909 )	5 years ago
Matt Keeler	8f0ab0129e	Miscellaneous Fixes (#6896 ) Ensure we close the Sentinel Evaluator so as not to leak go routines Fix a bunch of test logging so that various warnings when starting a test agent go to the ltest logger and not straight to stdout. Various canned ent meta types always return a valid pointer (no more nils). This allows us to blindly deref + assign in various places. Update ACL index tracking to ensure oss -> ent upgrades will work as expected. Update ent meta parsing to include function to disallow wildcarding.	5 years ago
Freddy	60f6ec0c2f	Store check type in catalog (#6561 )	5 years ago
R.B. Boyer	9566df524e	agent: cache notifications work after error if the underlying RPC returns index=1 (#6547 ) Fixes #6521 Ensure that initial failures to fetch an agent cache entry using the notify API where the underlying RPC returns a synthetic index of 1 correctly recovers when those RPCs resume working. The bug in the Cache.notifyBlockingQuery used to incorrectly "fix" the index for the next query from 0 to 1 for all queries, when it should have not done so for queries that errored. Also fixed some things that made debugging difficult: - config entry read/list endpoints send back QueryMeta headers - xds event loops don't swallow the cache notification errors	5 years ago
Freddy	fdd10dd8b8	Expose HTTP-based paths through Connect proxy (#6446 ) Fixes: #5396 This PR adds a proxy configuration stanza called expose. These flags register listeners in Connect sidecar proxies to allow requests to specific HTTP paths from outside of the node. This allows services to protect themselves by only listening on the loopback interface, while still accepting traffic from non Connect-enabled services. Under expose there is a boolean checks flag that would automatically expose all registered HTTP and gRPC check paths. This stanza also accepts a paths list to expose individual paths. The primary use case for this functionality would be to expose paths for third parties like Prometheus or the kubelet. Listeners for requests to exposed paths are be configured dynamically at run time. Any time a proxy, or check can be registered, a listener can also be created. In this initial implementation requests to these paths are not authenticated/encrypted.	5 years ago
R.B. Boyer	5882e21b2b	agent: tolerate more failure scenarios during service registration with central config enabled (#6472 ) Also: * Finished threading replaceExistingChecks setting (from GH-4905) through service manager. * Respected the original configSource value that was used to register a service or a check when restoring persisted data. * Run several existing tests with and without central config enabled (not exhaustive yet). * Switch to ioutil.ReadFile for all types of agent persistence.	5 years ago
R.B. Boyer	f9496dc627	sdk: add freelist tracking and ephemeral port range skipping to freeport This should cut down on test flakiness. Problems handled: - If you had enough parallel test cases running, the former circular approach to handling the port block could hand out the same port to multiple cases before they each had a chance to bind them, leading to one of the two tests to fail. - The freeport library would allocate out of the ephemeral port range. This has been corrected for Linux (which should cover CI). - The library now waits until a formerly-in-use port is verified to be free before putting it back into circulation.	5 years ago
Sarah Adams	001137e5e5	test: ensure all TestAgent constructions use a constructor (#6443 ) ensure all TestAgent constructions use a constructor to get start retries + test logs going to the right place Fixes #6435	5 years ago
Sarah Adams	74461406e0	remove funky panic/recover in agent tests (#6442 )	5 years ago
Sarah Adams	4ed5515fca	refactor & add better retry logic to NewTestAgent (#6363 ) Fixes #6361	5 years ago
Mike Morris	65be58703c	connect: remove managed proxies (#6220 ) * connect: remove managed proxies implementation and all supporting config options and structs * connect: remove deprecated ProxyDestination * command: remove CONNECT_PROXY_TOKEN env var * agent: remove entire proxyprocess proxy manager * test: remove all managed proxy tests * test: remove irrelevant managed proxy note from TestService_ServerTLSConfig * test: update ContentHash to reflect managed proxy removal * test: remove deprecated ProxyDestination test * telemetry: remove managed proxy note * http: remove /v1/agent/connect/proxy endpoint * ci: remove deprecated test exclusion * website: update managed proxies deprecation page to note removal * website: remove managed proxy configuration API docs * website: remove managed proxy note from built-in proxy config * website: add note on removing proxy subdirectory of data_dir	5 years ago
Paul Banks	f38da47c55	Allow raft TrailingLogs to be configured. (#6186 ) This fixes pathological cases where the write throughput and snapshot size are both so large that more than 10k log entries are written in the time it takes to restore the snapshot from disk. In this case followers that restart can never catch up with leader replication again and enter a loop of constantly downloading a full snapshot and restoring it only to find that snapshot is already out of date and the leader has truncated its logs so a new snapshot is sent etc. In general if you need to adjust this, you are probably abusing Consul for purposes outside its design envelope and should reconsider your usage to reduce data size and/or write volume.	5 years ago
R.B. Boyer	b16d7f00bc	agent: avoid reverting any check updates that occur while a service is being added or the config is reloaded (#6144 )	5 years ago
Hans Hasselberg	33a7df3330	tls: auto_encrypt enables automatic RPC cert provisioning for consul clients (#5597 )	5 years ago
Pierre Souchay	0e907f5aa8	Support for maximum size for Output of checks (#5233 ) * Support for maximum size for Output of checks This PR allows users to limit the size of output produced by checks at the agent and check level. When set at the agent level, it will limit the output for all checks monitored by the agent. When set at the check level, it can override the agent max for a specific check but only if it is lower than the agent max. Default value is 4k, and input must be at least 1.	5 years ago
R.B. Boyer	40336fd353	agent: fix several data races and bugs related to node-local alias checks (#5876 ) The observed bug was that a full restart of a consul datacenter (servers and clients) in conjunction with a restart of a connect-flavored application with bring-your-own-service-registration logic would very frequently cause the envoy sidecar service check to never reflect the aliased service. Over the course of investigation several bugs and unfortunate interactions were corrected: (1) local.CheckState objects were only shallow copied, but the key piece of data that gets read and updated is one of the things not copied (the underlying Check with a Status field). When the stock code was run with the race detector enabled this highly-relevant-to-the-test-scenario field was found to be racy. Changes: a) update the existing Clone method to include the Check field b) copy-on-write when those fields need to change rather than incrementally updating them in place. This made the observed behavior occur slightly less often. (2) If anything about how the runLocal method for node-local alias check logic was ever flawed, there was no fallback option. Those checks are purely edge-triggered and failure to properly notice a single edge transition would leave the alias check incorrect until the next flap of the aliased check. The change was to introduce a fallback timer to act as a control loop to double check the alias check matches the aliased check every minute (borrowing the duration from the non-local alias check logic body). This made the observed behavior eventually go away when it did occur. (3) Originally I thought there were two main actions involved in the data race: A. The act of adding the original check (from disk recovery) and its first health evaluation. B. The act of the HTTP API requests coming in and resetting the local state when re-registering the same services and checks. It took awhile for me to realize that there's a third action at work: C. The goroutines associated with the original check and the later checks. The actual sequence of actions that was causing the bad behavior was that the API actions result in the original check to be removed and re-added _without waiting for the original goroutine to terminate_. This means for brief windows of time during check definition edits there are two goroutines that can be sending updates for the alias check status. In extremely unlikely scenarios the original goroutine sees the aliased check start up in `critical` before being removed but does not get the notification about the nearly immediate update of that check to `passing`. This is interlaced wit the new goroutine coming up, initializing its base case to `passing` from the current state and then listening for new notifications of edge triggers. If the original goroutine "finishes" its update, it then commits one more write into the local state of `critical` and exits leaving the alias check no longer reflecting the underlying check. The correction here is to enforce that the old goroutines must terminate before spawning the new one for alias checks.	6 years ago
Alvin Huang	f45e495e38	Merge pull request #5376 from hashicorp/fix-tests Fix tests in prep for CircleCI Migration	6 years ago
Jeff Mitchell	4243c3ae42	Move internal/ to sdk/ (#5568 ) * Move internal/ to sdk/ * Add a readme to the SDK folder	6 years ago
Jeff Mitchell	47c390025b	Convert to Go Modules (#5517 ) * First conversion * Use serf 0.8.2 tag and associated updated deps * * Move freeport and testutil into internal/ * Make internal/ its own module * Update imports * Add replace statements so API and normal Consul code are self-referencing for ease of development * Adapt to newer goe/values * Bump to new cleanhttp * Fix ban nonprintable chars test * Update lock bad args test The error message when the duration cannot be parsed changed in Go 1.12 (ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test. * Update another test as well * Bump travis * Bump circleci * Bump go-discover and godo to get rid of launchpad dep * Bump dockerfile go version * fix tar command * Bump go-cleanhttp	6 years ago
Hans Hasselberg	e7134a0dab	agent: only use TestAgent when appropriate (#5502 )	6 years ago
Valentin Fritz	21f149de8b	Fix checks removal when removing service (#5457 ) Fix my recently discovered issue described here: #5456	6 years ago
Hans Hasselberg	7e11dd82aa	agent: enable reloading of tls config (#5419 ) This PR introduces reloading tls configuration. Consul will now be able to reload the TLS configuration which previously required a restart. It is not yet possible to turn TLS ON or OFF with these changes. Only when TLS is already turned on, the configuration can be reloaded. Most importantly the certificates and CAs.	6 years ago
Aestek	2aac4d5168	Register and deregisters services and their checks atomically in the local state (#5012 ) Prevent race between register and deregister requests by saving them together in the local state on registration. Also adds more cleaning in case of failure when registering services / checks.	6 years ago

1 2 3 4 5

223 Commits (3c3bdba926f97fa65fac00c50bf4ff081d4889d7)