Commit Graph

104 Commits (a1cb79b1f8f24f245a78d173613658b79f19b86b)

Author SHA1 Message Date
Daniel Nephin a583415bed Merge pull request #10161 from hashicorp/dnephin/update-deps
Update a couple dependencies
2021-05-04 18:32:22 +00:00
Paul Banks fa1b308c7b Make Raft trailing logs and snapshot timing reloadable (#10129)
* WIP reloadable raft config

* Pre-define new raft gauges

* Update go-metrics to change gauge reset behaviour

* Update raft to pull in new metric and reloadable config

* Add snapshot persistance timing and installSnapshot to our 'protected' list as they can be infrequent but are important

* Update telemetry docs

* Update config and telemetry docs

* Add note to oldestLogAge on when it is visible

* Add changelog entry

* Update website/content/docs/agent/options.mdx

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2021-05-04 14:40:40 +00:00
R.B. Boyer 6a39b47448 Support Incremental xDS mode (#9855)
This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged.

Union of all commit messages follows to give an overarching summary:

xds: exclusively support incremental xDS when using xDS v3

Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail.
Work around a strange older envoy behavior involving empty CDS responses over incremental xDS.
xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support

Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead.
Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client.
xds: pull out checkStreamACLs method in advance of a later commit

xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings

In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty.

This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10.

xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time
2021-04-29 18:54:53 +00:00
R.B. Boyer 2c3d7da5dd
mod: bump to github.com/hashicorp/mdns v1.0.4 (#10018) 2021-04-14 14:17:52 -05:00
Daniel Nephin 46279547ec Update memberlist to v0.2.3
To pickup data race fixes
2021-03-24 18:20:19 -04:00
Daniel Nephin cae4b2c0eb Update go-memdb
To use a version that will not panic when an iterator is used with modifications.
2021-01-28 17:19:55 -05:00
Daniel Nephin 727a402810
Merge pull request #9302 from hashicorp/dnephin/add-service-3
agent: remove ServiceManager.Start goroutine
2021-01-28 16:59:41 -05:00
Matt Keeler f561462064
Upgrade raft-autopilot and wait for autopilot it to stop when revoking leadership (#9644)
Fixes: 9626
2021-01-27 11:14:52 -05:00
Daniel Nephin 28736e60fd lib/mutex: add mutex with TryLock and update vendor 2021-01-25 18:01:47 -05:00
Daniel Nephin aeb9c09e25 Update mapstructure 2021-01-12 12:24:56 -05:00
Pierre Souchay 408b249fc5 [bugfix] Prometheus metrics without warnings
go-metrics is updated to 0.3.6 to properly handle help in prometheus metrics

This fixes https://github.com/hashicorp/consul/issues/9303 and
https://github.com/hashicorp/consul/issues/9471
2021-01-06 13:54:05 +01:00
Mike Morris ab927d5480 Merge pull request #9270 from hashicorp/release/1.9.0
merge: release/1.9.0 back into 1.9.x
2020-11-24 17:36:47 -05:00
Matt Keeler c048e86bb2
Switch to using the external autopilot module 2020-11-09 09:22:11 -05:00
Mike Morris 75019baadd
chore: upgrade to gopsutil/v3 (#9118)
* deps: update golang.org/x/sys

* deps: update imports to gopsutil/v3

* chore: make update-vendor
2020-11-06 20:48:38 -05:00
Kit Patella 9636906e53 rollback golang.org/x/sys version to fix distro-build 2020-11-05 12:09:07 -08:00
Kit Patella 9d9c4a646b upgrade go-metrics to latest 2020-11-04 14:02:13 -08:00
Kyle Havlovitz 4abe96aa74 vendor: Update github.com/hashicorp/yamux 2020-10-09 05:05:46 -07:00
Kyle Havlovitz dd6ed08924 vendor: Update github.com/hashicorp/mdns 2020-10-09 04:43:27 -07:00
Kyle Havlovitz 1cc012b202 vendor: Update github.com/hashicorp/hil 2020-10-09 04:43:27 -07:00
Kyle Havlovitz b95ab0d33c vendor: Update github.com/hashicorp/go-version 2020-10-09 04:43:27 -07:00
Kyle Havlovitz f389f1184d vendor: Update github.com/hashicorp/go-memdb 2020-10-09 04:43:27 -07:00
Kyle Havlovitz 40481e2b8f vendor: Update github.com/hashicorp/go-checkpoint 2020-10-09 04:43:27 -07:00
Mike Morris 708957a982
chore: update raft to v1.2.0 (#8822) 2020-10-08 15:07:10 -04:00
Matt Keeler 38f5ddce2a
Add per-agent reconnect timeouts (#8781)
This allows for client agent to be run in a more stateless manner where they may be abruptly terminated and not expected to come back. If advertising a per-agent reconnect timeout using the advertise_reconnect_timeout configuration when that agent leaves, other agents will wait only that amount of time for the agent to come back before reaping it.

This has the advantageous side effect of causing servers to deregister the node/services/checks for that agent sooner than if the global reconnect_timeout was used.
2020-10-08 15:02:19 -04:00
Mike Morris 1ebc2fb006
chore(deps): update gopsutil to v2.20.9 (#8843)
* core(deps): bump golang.org/x/sys

To resolve /go/pkg/mod/github.com/shirou/gopsutil@v2.20.9+incompatible/host/host_bsd.go:20:13: undefined: unix.SysctlTimeval

* chore(deps): make update-vendor
2020-10-07 12:57:18 -04:00
Daniel Nephin 627449a870 Vendor gofuzz and google/go-cmp 2020-09-28 18:28:37 -04:00
Kyle Havlovitz b1b21139ca Merge branch 'master' into vault-ca-renew-token 2020-09-15 14:39:04 -07:00
Kyle Havlovitz 1cd7c43544 Update vault CA for latest api client 2020-09-15 13:33:55 -07:00
Kyle Havlovitz 74dc50a771 vendor: Update vault api package 2020-09-15 12:45:29 -07:00
Daniel Nephin 0c87cf468c Update go-metrics dependencies, to use metrics.Default() 2020-09-14 19:05:22 -04:00
Mike Morris 13017e0af9 vendor: bump consul/api to v1.7.0 2020-09-10 21:40:41 -04:00
R.B. Boyer 74d5df7c7a
xds: use envoy's rbac filter to handle intentions entirely within envoy (#8569) 2020-08-27 12:20:58 -05:00
Hans Hasselberg a932aafc91
add primary keys to list keyring (#8522)
During gossip encryption key rotation it would be nice to be able to see if all nodes are using the same key. This PR adds another field to the json response from `GET v1/operator/keyring` which lists the primary keys in use per dc. That way an operator can tell when a key was successfully setup as primary key.

Based on https://github.com/hashicorp/serf/pull/611 to add primary key to list keyring output:

```json
[
  {
    "WAN": true,
    "Datacenter": "dc2",
    "Segment": "",
    "Keys": {
      "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 6,
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 6
    },
    "PrimaryKeys": {
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 6
    },
    "NumNodes": 6
  },
  {
    "WAN": false,
    "Datacenter": "dc2",
    "Segment": "",
    "Keys": {
      "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 8,
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8
    },
    "PrimaryKeys": {
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8
    },
    "NumNodes": 8
  },
  {
    "WAN": false,
    "Datacenter": "dc1",
    "Segment": "",
    "Keys": {
      "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 3,
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8
    },
    "PrimaryKeys": {
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8
    },
    "NumNodes": 8
  }
]
```

I intentionally did not change the CLI output because I didn't find a good way of displaying this information. There are a couple of options that we could implement later:
* add a flag to show the primary keys
* add a flag to show json output

Fixes #3393.
2020-08-18 09:50:24 +02:00
s-christoff 102b7e55da
Update Go-Metrics 0.3.4 (#8478) 2020-08-11 11:17:43 -05:00
Mike Morris 48e7c07cf9
api: bump consul/api to v1.6.0 and consul/sdk to v0.6.0 (#8460)
* api: bump consul/sdk dependency to v0.6.0

* api: bump dependency to v1.6.0
2020-08-07 17:26:05 -04:00
Kyle Havlovitz f4efd53d57 vendor: Update github.com/armon/go-metrics to v0.3.3 2020-07-23 11:37:33 -07:00
Matt Keeler a6a1a0e3d6
Update mapstructure to v1.3.3 (#8361)
This was done in preparation for another PR where I was running into https://github.com/mitchellh/mapstructure/issues/202 and implemented a fix for the library.
2020-07-22 15:13:21 -04:00
R.B. Boyer e853368c23
gossip: Avoid issue where two unique leave events for the same node could lead to infinite rebroadcast storms (#8343)
bump serf to v0.9.3 to include fix for https://github.com/hashicorp/serf/pull/606
2020-07-21 15:48:10 -05:00
Pierre Souchay 20d1ea7d2d
Upgrade go-connlimit to v0.3.0 / return http 429 on too many connections (#8221)
Fixes #7527

I want to highlight this and explain what I think the implications are and make sure we are aware:

* `HTTPConnStateFunc` closes the connection when it is beyond the limit. `Close` does not block.
* `HTTPConnStateFuncWithDefault429Handler(10 * time.Millisecond)` blocks until the following is done (worst case):
  1) `conn.SetDeadline(10*time.Millisecond)` so that
  2) `conn.Write(429error)` is guaranteed to timeout after 10ms, so that the http 429 can be written and 
  3) `conn.Close` can happen

The implication of this change is that accepting any new connection is worst case delayed by 10ms. But only after a client reached the limit already.
2020-07-03 09:25:07 +02:00
Hans Hasselberg 95c027a3ea
Update gopsutil (#8208)
https://github.com/shirou/gopsutil/pull/895 is merged and fixes our
problem. Time to update. Since there is no new version just yet,
updating to the sha.
2020-07-01 14:47:56 +02:00
Matt Keeler e9835610f3
Add a test for go routine leaks
This is in its own separate package so that it will be a separate test binary that runs thus isolating the go runtime from other tests and allowing accurate go routine leak checking.

This test would ideally use goleak.VerifyTestMain but that will fail 100% of the time due to some architectural things (blocking queries and net/rpc uncancellability).

This test is not comprehensive. We should enable/exercise more features and more cluster configurations. However its a start.
2020-06-24 17:09:50 -04:00
R.B. Boyer c63c994b04
connect: upgrade github.com/envoyproxy/go-control-plane to v0.9.5 (#8165) 2020-06-23 15:19:56 -05:00
Paul Banks f6ac08be04 state: track changes so that they may be used to produce change events 2020-06-16 13:04:29 -04:00
Daniel Nephin 98d271bee9 Update google.golang.org/api and stretchr/testify
To match the versions used in enterprise, should slightly reduce the
chances of getting a merge conflict when using `go.mod`.
2020-06-09 16:03:05 -04:00
Daniel Nephin db74f09b6b Update protobuf and golang.org/x/... vendor
Partially extracted from #7547

Updates protobuf to the most recent in the 1.3.x series, and updates
golang.org/x/sys to a7d97aace0b0 because of https://github.com/shirou/gopsutil/issues/853
prevents updating to a more recent version.

This breaking change in x/sys also prevents us from getting a newer
version of x/net. In the future, if gopsutil is not patched,  we may want to run a fork version of
gopsutil so that we can update both x/net and x/sys.
2020-06-09 14:46:41 -04:00
Daniel Nephin 99eb583ebc
Replace goe/verify.Values with testify/require.Equal (#7993)
* testing: replace most goe/verify.Values with require.Equal

One difference between these two comparisons is that go/verify considers
nil slices/maps to be equal to empty slices/maps, where as testify/require
does not, and does not appear to provide any way to enable that behaviour.

Because of this difference some expected values were changed from empty
slices to nil slices, and some calls to verify.Values were left.

* Remove github.com/pascaldekloe/goe/verify

Reduce the number of assertion packages we use from 2 to 1
2020-06-02 12:41:25 -04:00
R.B. Boyer 1efafd7523
acl: add auth method for JWTs (#7846) 2020-05-11 20:59:29 -05:00
Mike Morris 291e6af33a
vendor: revert golang.org/x/sys bump to avoid FreeBSD regression (#7780) 2020-05-05 09:26:17 +02:00
Hans Hasselberg c4093c87cc
agent: don't let left nodes hold onto their node-id (#7747) 2020-05-04 18:39:08 +02:00
Matt Keeler daec810e34
Merge pull request #7714 from hashicorp/oss-sync/msp-agent-token 2020-05-04 11:33:50 -04:00