Commit Graph

1240 Commits (98d95cfa8045dd71c3bcd29ec25e77ac8d2c080b)

Author SHA1 Message Date
Matt Keeler e81c85c051
Fix #4515: Segfault when serf_wan port was -1 but reconnect_time_wan was set (#4531)
Fixes #4515 

This just slightly refactors the logic to only attempt to set the serf wan reconnect timeout when the rest of the serf wan settings are configured - thus avoiding a segfault.
2018-08-17 14:44:25 -04:00
Kyle Havlovitz e5e1f867e5
Merge branch 'master' into ca-snapshot-fix 2018-08-16 13:00:54 -07:00
Kyle Havlovitz f186edc42c
fsm: add connect service config to snapshot/restore test 2018-08-16 12:58:54 -07:00
nickmy9729 beddf03b26 Added code to allow snapshot inclusion of NodeMeta (#4527) 2018-08-16 15:33:35 -04:00
Kyle Havlovitz b51d76f469
fsm: add missing CA config to snapshot/restore logic 2018-08-16 11:58:50 -07:00
Kyle Havlovitz 4b35d877ca
autopilot: don't follow the normal server removal rules for nonvoters 2018-08-14 14:24:51 -07:00
Kyle Havlovitz ea14482376
Fix stats fetcher healthcheck RPCs not being independent 2018-08-14 14:23:52 -07:00
Pierre Souchay 0d6de257a2 Display more information about check being not properly added when it fails (#4405)
* Display more information about check being not properly added when it fails

It follows an incident where we add lots of error messages:

  [WARN] consul.fsm: EnsureRegistration failed: failed inserting check: Missing service registration

That seems related to Consul failing to restart on respective agents.

Having Node information as well as service information would help diagnose the issue.

* Renamed ensureCheckIfNodeMatches() as requested by @banks
2018-08-14 17:45:33 +01:00
Freddy 6d43d24edb
Improve reliability of tests with TestAgent (#4525)
- Add WaitForTestAgent to tests flaky due to missing serfHealth registration

- Fix bug in retries calling Fatalf with *testing.T

- Convert TestLockCommand_ChildExitCode to table driven test
2018-08-14 12:08:33 -04:00
Pierre Souchay ef3b81ab13 Allow to rename nodes with IDs, will fix #3974 and #4413 (#4415)
* Allow to rename nodes with IDs, will fix #3974 and #4413

This change allow to rename any well behaving recent agent with an
ID to be renamed safely, ie: without taking the name of another one
with case insensitive comparison.

Deprecated behaviour warning
----------------------------

Due to asceding compatibility, it is still possible however to
"take" the name of another name by not providing any ID.

Note that when not providing any ID, it is possible to have 2 nodes
having similar names with case differences, ie: myNode and mynode
which might lead to DB corruption on Consul server side and
lead to server not properly restarting.

See #3983 and #4399 for Context about this change.

Disabling registration of nodes without IDs as specified in #4414
should probably be the way to go eventually.

* Removed the case-insensitive search when adding a node within the else
block since it breaks the test TestAgentAntiEntropy_Services

While the else case is probably legit, it will be fixed with #4414 in
a later release.

* Added again the test in the else to avoid duplicated names, but
enforce this test only for nodes having IDs.

Thus most tests without any ID will work, and allows us fixing

* Added more tests regarding request with/without IDs.

`TestStateStore_EnsureNode` now test registration and renaming with IDs

`TestStateStore_EnsureNodeDeprecated` tests registration without IDs
and tests removing an ID from a node as well as updated a node
without its ID (deprecated behaviour kept for backwards compatibility)

* Do not allow renaming in case of conflict, including when other node has no ID

* Fixed function GetNodeID that was not working due to wrong type when searching node from its ID

Thus, all tests about renaming were not working properly.

Added the full test cas that allowed me to detect it.

* Better error messages, more tests when nodeID is not a valid UUID in GetNodeID()

* Added separate TestStateStore_GetNodeID to test GetNodeID.

More complete test coverage for GetNodeID

* Added new unit test `TestStateStore_ensureNoNodeWithSimilarNameTxn`

Also fixed comments to be clearer after remarks from @banks

* Fixed error message in unit test to match test case

* Use uuid.ParseUUID to parse Node.ID as requested by @mkeeler
2018-08-10 11:30:45 -04:00
Siva Prasad c88900aaa9
PR to fix TestAgent_IndexChurn and TestPreparedQuery_Wrapper. (#4512)
* Fixes TestAgent_IndexChurn

* Fixes TestPreparedQuery_Wrapper

* Increased sleep in agent_test for IndexChurn to 500ms

* Made the comment about joinWAN operation much less of a cliffhanger
2018-08-09 12:40:07 -04:00
Armon Dadgar 4f1fd34e9e consul: Update buffer sizes 2018-08-08 10:26:58 -07:00
Siva Prasad 288d350a73
Revert "CA initialization while boostrapping and TestLeader_ChangeServerID fix." (#4497)
* Revert "BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472)"

This reverts commit cec5d72396.

* Revert "CA initialization while boostrapping and TestLeader_ChangeServerID fix. (#4493)"

This reverts commit 589b589b53.
2018-08-07 08:29:48 -04:00
Pierre Souchay cec5d72396 BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472)
- Improve resilience of testrpc.WaitForLeader()

- Add additionall retry to CI

- Increase "go test" timeout to 8m

- Add wait for cluster leader to several tests in the agent package

- Add retry to some tests in the api and command packages
2018-08-06 19:46:09 -04:00
Siva Prasad 589b589b53
CA initialization while boostrapping and TestLeader_ChangeServerID fix. (#4493)
* connect: fix an issue with Consul CA bootstrapping being interrupted

* streamline change server id test
2018-08-06 16:15:24 -04:00
Siva Prasad 865068a358
DNS : Fixes recursors answering the DNS query to properly return the correct response. (#4461)
* Fixes the DNS recursor properly resolving the requests

* Added a test case for the recursor bug

* Refactored code && added a test case for all failing recursors

* Inner indentation moved into else if check
2018-08-02 10:12:52 -04:00
Paul Banks 71dd3b408a
Fixes memory leak when blocking on /event/list (#4482) 2018-08-02 14:54:48 +01:00
mkeeler e716d1b5f8
Release v1.2.2 2018-07-30 16:01:13 +00:00
Matt Keeler 870a6ad6a8
Handle resolving proxy tokens when parsing HTTP requests (#4453)
Fixes: #4441

This fixes the issue with Connect Managed Proxies + ACLs being broken.

The underlying problem was that the token parsed for most http endpoints was sent untouched to the servers via the RPC request. These changes make it so that at the HTTP endpoint when parsing the token we additionally attempt to convert potential proxy tokens into regular tokens before sending to the RPC endpoint. Proxy tokens are only valid on the agent with the managed proxy so the resolution has to happen before it gets forwarded anywhere.
2018-07-30 09:11:51 -04:00
Matt Keeler 0e0227792b
Gossip tuneables (#4444)
Expose a few gossip tuneables for both lan and wan interfaces

gossip_nodes
gossip_interval
probe_timeout
probe_interval
retransmit_mult
suspicion_mult
2018-07-26 11:39:49 -04:00
Kyle Havlovitz fa0d8aff33
fix inconsistency in TestConnectCAConfig_GetSet 2018-07-26 07:46:47 -07:00
Paul Banks 8dd50d5b2d
Add config option to disable HTTP printable char path check (#4442) 2018-07-26 13:53:39 +01:00
Kyle Havlovitz ed87949385
Merge pull request #4400 from hashicorp/leaf-cert-ttl
Add configurable leaf cert TTL to Connect CA
2018-07-25 17:53:25 -07:00
Kyle Havlovitz f67a4d59c0
connect/ca: simplify passing of leaf cert TTL 2018-07-25 17:51:45 -07:00
Siva Prasad f4a1c381a5 Vendoring update for go-discover. (#4412)
* New Providers added and updated vendoring for go-discover

* Vendor.json formatted using make vendorfmt

* Docs/Agent/auto-join: Added documentation for the new providers introduced in this PR

* Updated the golang.org/x/sys/unix in the vendor directory

* Agent: TestGoDiscoverRegistration updated to reflect the addition of new providers

* Deleted terraform.tfstate from vendor.

* Deleted terraform.tfstate.backup

Deleted terraform state file artifacts from unknown runs.

* Updated x/sys/windows vendor for Windows binary compilation
2018-07-25 16:21:04 -07:00
Paul Banks 8cbeb29e73
Fixes #4421: General solution to stop blocking queries with index 0 (#4437)
* Fix theoretical cache collision bug if/when we use more cache types with same result type

* Generalized fix for blocking query handling when state store methods return zero index

* Refactor test retry to only affect CI

* Undo make file merge

* Add hint to error message returned to end-user requests if Connect is not enabled when they try to request cert

* Explicit error for Roots endpoint if connect is disabled

* Fix tests that were asserting old behaviour
2018-07-25 20:26:27 +01:00
Paul Banks 5635227fa6
Allow config-file based Service Definitions for unmanaged proxies and Connect-natice apps. (#4443) 2018-07-25 19:55:41 +01:00
Paul Banks d5e934f9ff
Ooops that was meant to be to a branch no master... EMORECOFFEE
Revert "Add config option to disable HTTP printable char path check"

This reverts commit eebe45a47b.
2018-07-25 15:54:11 +01:00
Paul Banks eebe45a47b
Add config option to disable HTTP printable char path check 2018-07-25 15:52:37 +01:00
Paul Banks e954450dec
Merge pull request #4353 from azam/add-serf-lan-wan-port-args
Make RPC, Serf LAN, Serf WAN port configurable from CLI
2018-07-24 12:33:10 +01:00
Kyle Havlovitz ce10de036e
connect/ca: check LeafCertTTL when rotating expired roots 2018-07-20 16:04:04 -07:00
Mitchell Hashimoto 7fa6bb022f
Merge pull request #4320 from hashicorp/f-alias-check
Add "Alias" Check Type
2018-07-20 13:01:33 -05:00
azam 342bcb1c24 Make Serf LAN & WAN port configurable from CLI
Make RPC port accessible to CLI

Add tests and documentation for server-port, serf-lan-port, serf-wan-port CLI arguments
2018-07-21 02:17:21 +09:00
Mitchell Hashimoto b3854fdd28
agent/local: silly spacing on select statements 2018-07-19 14:21:30 -05:00
Mitchell Hashimoto 8c72bb0cdf
agent/local: address remaining test feedback 2018-07-19 14:20:50 -05:00
Matt Keeler 560c9c26f7 Use the agent logger instead of log module 2018-07-19 11:22:01 -04:00
Matt Keeler ca5851318d Update a couple erroneous tests. 2018-07-19 09:20:51 -04:00
Mitchell Hashimoto 9f128e40d6
agent/local: don't use time.After in test since notify is instant 2018-07-18 16:16:28 -05:00
Matt Keeler 3fe5f566f2 Persist proxies from config files
Also change how loadProxies works. Now it will load all persisted proxies into a map, then when loading config file proxies will look up the previous proxy token in that map.
2018-07-18 17:04:35 -04:00
Kyle Havlovitz d6ca015a42
connect/ca: add configurable leaf cert TTL 2018-07-16 13:33:37 -07:00
Matt Keeler c891e264ca Fix issue with choosing a client addr that is 0.0.0.0 or :: 2018-07-16 16:30:15 -04:00
Mitchell Hashimoto 9a90400821
agent/checks: prevent overflow of backoff 2018-07-12 10:21:49 -07:00
Mitchell Hashimoto d6ecd97d1d
agent: use the correct ACL token for alias checks 2018-07-12 10:17:53 -07:00
Mitchell Hashimoto f97bfd5be8
agent: address some basic feedback 2018-07-12 09:36:11 -07:00
Mitchell Hashimoto 19ced12668
agent: alias checks have no interval 2018-07-12 09:36:11 -07:00
Mitchell Hashimoto 5bc27feb0b
agent/structs: check is alias if node is empty 2018-07-12 09:36:11 -07:00
Mitchell Hashimoto 36e330941a
agent/checks: support node-only checks 2018-07-12 09:36:11 -07:00
Mitchell Hashimoto 1e9233eec1
agent/checks: set critical if RPC fails 2018-07-12 09:36:11 -07:00
Mitchell Hashimoto e9914ee71c
agent/checks: use local state for local services 2018-07-12 09:36:11 -07:00
Mitchell Hashimoto 7543d270e2
agent/local: support local alias checks 2018-07-12 09:36:10 -07:00
Mitchell Hashimoto 4a67beb734
agent: run alias checks 2018-07-12 09:36:10 -07:00
Mitchell Hashimoto 60c75b88da
agent/checks: reflect node failure as alias check failure 2018-07-12 09:36:10 -07:00
Mitchell Hashimoto f0658a0ede
agent/config: support configuring alias check 2018-07-12 09:36:10 -07:00
Mitchell Hashimoto 632e4a2c69
agent/checks: add Alias check type 2018-07-12 09:36:09 -07:00
mkeeler 39f93f011e
Release v1.2.1 2018-07-12 16:33:56 +00:00
Matt Keeler 63d5c069fc
Merge pull request #4379 from hashicorp/persist-intermediates
connect: persist intermediate CAs on leader change
2018-07-12 12:09:13 -04:00
Paul Banks 9015cd62ab
Merge pull request #4381 from hashicorp/proxy-check-default
Proxy check default
2018-07-12 17:08:35 +01:00
Matt Keeler 0e83059d1f
Revert "Allow changing Node names since Node now have IDs" 2018-07-12 11:19:21 -04:00
Matt Keeler 91150cca59 Fixup formatting 2018-07-12 10:14:26 -04:00
Matt Keeler 3807e04de9 Revert PR 4294 - Catalog Register: Generate UUID for services registered without one
UUID auto-generation here causes trouble in a few cases. The biggest being older
nodes reregistering will fail when the UUIDs are different and the names match

This reverts commit 0f70034082.
This reverts commit d1a8f9cb3f.
This reverts commit cf69ec42a4.
2018-07-12 10:06:50 -04:00
Matt Keeler 7572ca0f37
Merge pull request #4374 from hashicorp/feature/proxy-env-vars
Setup managed proxy environment with API client env vars
2018-07-12 09:13:54 -04:00
Paul Banks 8405b41f2b
Update proxy config docs and add test for ipv6 2018-07-12 13:07:48 +01:00
Paul Banks bb9a5c703b
Default managed proxy TCP check address sanely when proxy is bound to 0.0.0.0.
This also provides a mechanism to configure custom address or disable the check entirely from managed proxy config.
2018-07-12 12:57:10 +01:00
Matt Keeler 0f56ed2d01 Set api.Config’s InsecureSkipVerify to the value of !RuntimeConfig.VerifyOutgoing 2018-07-12 07:49:23 -04:00
Matt Keeler 22e4058893 Use type switch instead of .Network for more reliably detecting UnixAddrs 2018-07-12 07:30:17 -04:00
Matt Keeler 700a275ddf Look specifically for tcp instead of unix
Add runtime -> api.Config tests
2018-07-11 17:25:36 -04:00
Matt Keeler c8df4b824c Update proxy manager test - test passing ProxyEnv vars 2018-07-11 16:50:27 -04:00
Kyle Havlovitz f95c6807e7
connect: use reflect.DeepEqual instead for test 2018-07-11 13:10:58 -07:00
Matt Keeler 98ead2a8f8
Merge pull request #3983 from pierresouchay/node_renaming
Allow changing Node names since Node now have IDs
2018-07-11 16:03:02 -04:00
Kyle Havlovitz 4e5fb6bc19
connect: add provider state to snapshots 2018-07-11 11:34:49 -07:00
Kyle Havlovitz 462ace4867
connect: update leader initializeCA comment 2018-07-11 10:00:42 -07:00
Kyle Havlovitz 1d3f4b5099
connect: persist intermediate CAs on leader change 2018-07-11 09:44:30 -07:00
Matt Keeler c54b43bef3 PR Updates
Proxy now doesn’t need to know anything about the api as we pass env vars to it instead of the api config.
2018-07-11 09:44:54 -04:00
Matt Keeler 4d1ead10b3
Merge pull request #4371 from hashicorp/bugfix/gh-4358
Remove https://prefix from TLSConfig.Address
2018-07-11 08:50:10 -04:00
Pierre Souchay fecae3de21 When renaming a node, ensure the name is not taken by another node.
Since DNS is case insensitive and DB as issues when similar names with different
cases are added, check for unicity based on case insensitivity.

Following another big incident we had in our cluster, we also validate
that adding/renaming a not does not conflicts with case insensitive
matches.

We had the following error once:

 - one node called: mymachine.MYDC.mydomain was shut off
 - another node (different ID) was added with name: mymachine.mydc.mydomain before
   72 hours

When restarting the consul server of domain, the consul server restarted failed
to start since it detected an issue in RAFT database because
mymachine.MYDC.mydomain and mymachine.mydc.mydomain had the same names.

Checking at registration time with case insensitivity should definitly fix
those issues and avoid Consul DB corruption.
2018-07-11 14:42:54 +02:00
Matt Keeler bd76a34002
Merge pull request #4365 from pierresouchay/fix_test_warning
Fixed compilation warning about wrong type
2018-07-10 16:53:29 -04:00
Matt Keeler 3b6eef8ec6 Pass around an API Config object and convert to env vars for the managed proxy 2018-07-10 12:13:51 -04:00
Pierre Souchay 7d2e4b77ec Use %q, not %s as it used to 2018-07-10 16:52:08 +02:00
Matt Keeler 0fd7e97c2d Merge remote-tracking branch 'origin/master' into bugfix/prevent-multi-cname 2018-07-10 10:26:45 -04:00
Matt Keeler d19c7d8882
Merge pull request #4303 from pierresouchay/non_blocking_acl
Only send one single ACL cache refresh across network when TTL is over
2018-07-10 08:57:33 -04:00
Matt Keeler d066fb7b18
Merge pull request #4362 from hashicorp/bugfix/gh-4354
Ensure TXT RRs always end up in the Additional section except for ANY or TXT queries
2018-07-10 08:50:31 -04:00
Pierre Souchay b112bdd52d Fixed compilation warning about wrong type
It fixes the following warnings:

  agent/config/builder.go:1201: Errorf format %q has arg s of wrong type *string
  agent/config/builder.go:1240: Errorf format %q has arg s of wrong type *string
2018-07-09 23:43:56 +02:00
Paul Banks 41c3a4ac8e
Merge pull request #4038 from pierresouchay/ACL_additional_info
Track calls blocked by ACLs using metrics
2018-07-09 20:21:21 +01:00
MagnumOpus21 371f0c3d5f Tests/Proxy : Changed function name to match the system being tested. 2018-07-09 13:18:57 -04:00
MagnumOpus21 9d57b72e81 Resolved merge conflicts 2018-07-09 12:48:34 -04:00
MagnumOpus21 300330e24b Agent/Proxy: Formatting and test cases fix 2018-07-09 12:46:10 -04:00
Matt Keeler 962f6a1816 Remove https://prefix from TLSConfig.Address 2018-07-09 12:31:15 -04:00
Matt Keeler cbf8f14451 Ensure TXT RRs always end up in the Additional section except for ANY or TXT queries
This also changes where the enforcement of the enable_additional_node_meta_txt configuration gets applied.

formatNodeRecord returns the main RRs and the meta/TXT RRs in separate slices. Its then up to the caller to add to the appropriate sections or not.
2018-07-09 12:30:11 -04:00
MagnumOpus21 94e8ff55cf Proxy/Tests: Added test cases to check env variables 2018-07-09 12:28:29 -04:00
MagnumOpus21 6cecf2961d Agent/Proxy : Properly passes env variables to child 2018-07-09 12:28:29 -04:00
Pierre Souchay ff53648df2 Merge remote-tracking branch 'origin/master' into ACL_additional_info 2018-07-07 14:09:18 +02:00
Pierre Souchay 0e4e451a56 Fixed indentation in test 2018-07-07 14:03:34 +02:00
Kyle Havlovitz 401b206a2e
Store the time CARoot is rotated out instead of when to prune 2018-07-06 16:05:25 -07:00
MagnumOpus21 1cd1b55682 Agent/Proxy : Properly passes env variables to child 2018-07-05 22:04:29 -04:00
Matt Keeler e3783a75e7 Refactor to make this much less confusing 2018-07-03 11:04:19 -04:00
Matt Keeler 554035974e Add a bunch of comments about preventing multi-cname
Hopefully this a bit clearer as to the reasoning
2018-07-03 10:32:52 -04:00
Matt Keeler 22c2be5bf1 Fix some edge cases and add some tests. 2018-07-02 16:58:52 -04:00
Matt Keeler 9a8500412b Only allow 1 CNAME when querying for a service.
This just makes sure that if multiple services are registered with unique service addresses that we don’t blast back multiple CNAMEs for the same service DNS name and keeps us within the DNS specs.
2018-07-02 16:12:06 -04:00
Kyle Havlovitz 1492243e0a
connect/ca: add logic for pruning old stale RootCA entries 2018-07-02 10:35:05 -07:00
Matt Keeler 8a12d803fd
Merge pull request #4315 from hashicorp/bugfix/fix-server-enterprise
Move starting enterprise functionality
2018-07-02 12:28:10 -04:00