consul

Commit Graph

Author	SHA1	Message	Date
Dan Upton	d4c435856b	grpc: `protoc` plugin for generating gRPC rate limit specifications (#15564 ) Adds automation for generating the map of `gRPC Method Name → Rate Limit Type` used by the middleware introduced in #15550, and will ensure we don't forget to add new endpoints. Engineers must annotate their RPCs in the proto file like so: ``` rpc Foo(FooRequest) returns (FooResponse) { option (consul.internal.ratelimit.spec) = { operation_type: READ, }; } ``` When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will be installed that writes rate-limit specs as a JSON array to a file called `.ratelimit.tmp` (one per protobuf package/directory). After running Buf, `make proto` will execute a post-process script that will ingest all of the `.ratelimit.tmp` files and generate a Go file containing the mappings in the `agent/grpc-middleware` package. In the enterprise repository, it will write an additional file with the enterprise-only endpoints. If an engineer forgets to add the annotation to a new RPC, the plugin will return an error like so: ``` RPC Foo is missing rate-limit specification, fix it with: import "proto-public/annotations/ratelimit/ratelimit.proto"; service Bar { rpc Foo(...) returns (...) { option (hashicorp.consul.internal.ratelimit.spec) = { operation_type: OPERATION_READ \| OPERATION_WRITE \| OPERATION_EXEMPT, }; } } ``` In the future, this annotation can be extended to support rate-limit category (e.g. KV vs Catalog) and to determine the retry policy.	2 years ago
Semir Patel	bafa5c7156	Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700 )	2 years ago
Dan Stough	98ef5f28dd	[OSS] security: update x/net module (#15737 ) Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
cskh	36f05bc8fb	integ-test: test consul upgrade from the snapshot of a running cluster (#15595 ) * integ-test: test consul upgrade from the snapshot of a running cluster * use Target version as default Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	2 years ago
Dan Stough	227fd14287	chore: updates from 1.14.2 release (#15633 ) * chore: updates from 1.14.2 release	2 years ago
Derek Menteer	dc27e35f82	Consul 1.14 post-release updates (#15382 ) * Update changelog with 1.14 notes. * gomod version bumps for 1.14 release.	2 years ago
Chris S. Kim	985a4ee1b1	Update hcp-scada-provider to fix diamond dependency problem with go-msgpack (#15185 )	2 years ago
Dan Stough	05e93f7569	test: refactor testcontainers and add peering integ tests (#15084 )	2 years ago
boruszak	38b1a515f1	Mergimg	2 years ago
John Murret	79a541fd7d	Upgrade serf to v0.10.1 and memberlist to v0.5.0 to get memberlist size metrics and broadcast queue depth metric (#14873 ) * updating to serf v0.10.1 and memberlist v0.5.0 to get memberlist size metrics and memberlist broadcast queue depth metric * update changelog * update changelog * correcting changelog * adding "QueueCheckInterval" for memberlist to test * updating integration test containers to grab latest api	2 years ago
DanStough	7704daaad5	release updates for 1.13.2, 1.12.5, and 1.11.9	2 years ago
R.B. Boyer	a1e911a70c	tests: ensure integration tests show logs from the containers to help debugging (#13593 )	2 years ago
Dhia Ayachi	355cbfa766	update github.com/containerd/containerd to 1.5.13 (#13520 )	2 years ago
R.B. Boyer	fbee9eda08	test: break dep on main consul module (#13373 ) The main consul module is not a great library and complicates some oss/ent module issues. This undoes #13371	3 years ago
R.B. Boyer	979b9312ca	test: use a go mod replace trick for the compat test dependency on the main repo (#13371 )	3 years ago
cskh	74158a8aa2	Add isLeader metric to track if a server is a leader (#13304 ) CTIA-21: sdd is_leader metric to track if a server is a leader Co-authored-by: alex <8968914+acpana@users.noreply.github.com>	3 years ago
Dhia Ayachi	4488b6c339	Add versions compatibility tests between Consul (#12702 ) * add a sample * Consul cluster test * add build dockerfile * add tests to cover mixed versions tests * use flag to pass docker image name * remove default config and rely on flags to inject the right image to test * add cluster abstraction * fix imports and remove old files * fix imports and remove old files * fix dockerIgnore * make a `Node interface` and encapsulate ConsulContainer * fix a test bug where we only check the leader against a single node. * add upgrade tests to CI * fix yaml alignment * fix alignment take 2 * fix flag naming * fix image to build * fix test run and go mod tidy * add a debug command * run without RYUK * fix parallel run * add skip reaper code * make tempdir in local dir * chmod the temp dir to 0777 * chmod the right dir name * change executor to use machine instead of docker * add docker layer caching * remove setup docker * add gotestsum * install go version * use variable for GO installed version * add environment * add environment in the right place * do not disable RYUK in CI * add service check to tests * assertions outside routines * add queryBackend to the api query meta. * check if we are using the right backend for those tests (streaming) * change the tested endpoint to use one that have streaming. * refactor to test multiple scenarios for streaming * Fix dockerfile Co-authored-by: FFMMM <FFMMM@users.noreply.github.com> * rename Clients to clients Co-authored-by: FFMMM <FFMMM@users.noreply.github.com> * check if cluster have 0 node * tidy code and add some doc strings * use uuid instead of random string * add doc strings to tests * add queryBackend to the api query meta. * add a changelog * fix for api backend query * add missing require * fix q.QueryBackend * Revert "fix q.QueryBackend" This reverts commit `cd0e5f7b1a`. * fix circle ci config * tidy go mod after merging main * rename package and fix test scenario * update go download url * address review comments * rename flag in CI * add readme to the upgrade tests * fix golang download url * fix golang arch downloaded * fix AddNodes to handle an empty cluster case * use `parseBool` * rename circle job and add comment * update testcontainer to 0.13 * fix circle ci config * remove build docker file and use `make dev-docker` instead * Apply suggestions from code review Co-authored-by: Dan Upton <daniel@floppy.co> * fix a typo Co-authored-by: FFMMM <FFMMM@users.noreply.github.com> Co-authored-by: Dan Upton <daniel@floppy.co>	3 years ago
Dhia Ayachi	b83a790927	update raft to v1.3.8 (#12844 ) * update raft to v1.3.7 * add changelog * fix compilation error * fix HeartbeatTimeout * fix ElectionTimeout to reload only if value is valid * fix default values for `ElectionTimeout` and `HeartbeatTimeout` * fix test defaults * bump raft to v1.3.8	3 years ago
R.B. Boyer	f507f62f3c	peering: initial sync (#12842 ) - Add endpoints related to peering: read, list, generate token, initiate peering - Update node/service/check table indexing to account for peers - Foundational changes for pushing service updates to a peer - Plumb peer name through Health.ServiceNodes path see: ENT-1765, ENT-1280, ENT-1283, ENT-1283, ENT-1756, ENT-1739, ENT-1750, ENT-1679, ENT-1709, ENT-1704, ENT-1690, ENT-1689, ENT-1702, ENT-1701, ENT-1683, ENT-1663, ENT-1650, ENT-1678, ENT-1628, ENT-1658, ENT-1640, ENT-1637, ENT-1597, ENT-1634, ENT-1613, ENT-1616, ENT-1617, ENT-1591, ENT-1588, ENT-1596, ENT-1572, ENT-1555 Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: freddygv <freddy@hashicorp.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Evan Culver <eculver@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>	3 years ago
DanStough	95250e7915	Update go version to 1.18.1	3 years ago
R.B. Boyer	8beaca4e01	deps: update to latest go-discover (#12739 ) Fixes #11253 $ go mod why -m github.com/dgrijalva/jwt-go # github.com/dgrijalva/jwt-go (main module does not need module github.com/dgrijalva/jwt-go) $ go mod why -m github.com/form3tech-oss/jwt-go # github.com/form3tech-oss/jwt-go github.com/hashicorp/consul/agent github.com/hashicorp/go-discover github.com/hashicorp/go-discover/provider/azure github.com/Azure/go-autorest/autorest github.com/Azure/go-autorest/autorest/adal github.com/form3tech-oss/jwt-go	3 years ago
Matt Keeler	a553982506	Enable running autopilot state updates on all servers (#12617 ) * Fixes a lint warning about t.Errorf not supporting %w * Enable running autopilot on all servers On the non-leader servers all they do is update the state and do not attempt any modifications. * Fix the RPC conn limiting tests Technically they were relying on racey behavior before. Now they should be reliable.	3 years ago
Eric	e4b4f175ed	Bump go-control-plane * `go get cloud.google.com/go@v0.59.0` * `go get github.com/envoyproxy/go-control-plane@v0.9.9` * `make envoy-library` * Bumpprotoc to 3.15.8	3 years ago
Matt Keeler	15ddbbc686	Update raft-boltdb to pull in new writeCapacity metric (#12646 )	3 years ago
Eric	776f5843d0	remove gogo from pbservice	3 years ago
FFMMM	db27ea3484	[sync oss] add net/rpc interceptor implementation (#12573 ) * sync ent changes from 866dcb0667 Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> * update oss go.mod Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>	3 years ago
R.B. Boyer	58e053c336	raft: upgrade to v1.3.6 (#12496 ) Add additional protections on the Consul side to prevent NonVoters from bootstrapping raft. This should un-flake TestServer_Expect_NonVoters	3 years ago
R.B. Boyer	7b0548dd8d	server: suppress spurious blocking query returns where multiple config entries are involved (#12362 ) Starting from and extending the mechanism introduced in #12110 we can specially handle the 3 main special Consul RPC endpoints that react to many config entries in a single blocking query in Connect: - `DiscoveryChain.Get` - `ConfigEntry.ResolveServiceConfig` - `Intentions.Match` All of these will internally watch for many config entries, and at least one of those will likely be not found in any given query. Because these are blends of multiple reads the exact solution from #12110 isn't perfectly aligned, but we can tweak the approach slightly and regain the utility of that mechanism. ### No Config Entries Found In this case, despite looking for many config entries none may be found at all. Unlike #12110 in this scenario we do not return an empty reply to the caller, but instead synthesize a struct from default values to return. This can be handled nearly identically to #12110 with the first 1-2 replies being non-empty payloads followed by the standard spurious wakeup suppression mechanism from #12110. ### No Change Since Last Wakeup Once a blocking query loop on the server has completed and slept at least once, there is a further optimization we can make here to detect if any of the config entries that were present at specific versions for the prior execution of the loop are identical for the loop we just woke up for. In that scenario we can return a slightly different internal sentinel error and basically externally handle it similar to #12110. This would mean that even if 20 discovery chain read RPC handling goroutines wakeup due to the creation of an unrelated config entry, the only ones that will terminate and reply with a blob of data are those that genuinely have new data to report. ### Extra Endpoints Since this pattern is pretty reusable, other key config-entry-adjacent endpoints used by `agent/proxycfg` also were updated: - `ConfigEntry.List` - `Internal.IntentionUpstreams` (tproxy)	3 years ago
Dhia Ayachi	cd9d8d44a5	file watcher to be used for configuration auto-reload feature (#12301 ) * add config watcher to the config package * add logging to watcher * add test and refactor to add WatcherEvent. * add all API calls and fix a bug with recreated files * add tests for watcher * remove the unnecessary use of context * Add debug log and a test for file rename * use inode to detect if the file is recreated/replaced and only listen to create events. * tidy ups (#1535) * tidy ups * Add tests for inode reconcile * fix linux vs windows syscall * fix linux vs windows syscall * fix windows compile error * increase timeout * use ctime ID * remove remove/creation test as it's a use case that fail in linux * fix linux/windows to use Ino/CreationTime * fix the watcher to only overwrite current file id * fix linter error * fix remove/create test * set reconcile loop to 200 Milliseconds * fix watcher to not trigger event on remove, add more tests * on a remove event try to add the file back to the watcher and trigger the handler if success * fix race condition * fix flaky test * fix race conditions * set level to info * fix when file is removed and get an event for it after * fix to trigger handler when we get a remove but re-add fail * fix error message * add tests for directory watch and fixes * detect if a file is a symlink and return an error on Add * rename Watcher to FileWatcher and remove symlink deref * add fsnotify@v1.5.1 * fix go mod * fix flaky test * Apply suggestions from code review Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> * fix a possible stack overflow * do not reset timer on errors, rename OS specific files * start the watcher when creating it * fix data race in tests * rename New func * do not call handler when a remove event happen * events trigger on write and rename * fix watcher tests * make handler async * remove recursive call * do not produce events for sub directories * trim "/" at the end of a directory when adding * add missing test * fix logging * add todo * fix failing test * fix flaking tests * fix flaky test * add logs * fix log text * increase timeout * reconcile when remove * check reconcile when removed * fix reconcile move test * fix logging * delete invalid file * Apply suggestions from code review Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> * fix review comments * fix is watched to properly catch a remove * change test timeout * fix test and rename id * fix test to create files with different mod time. * fix deadlock when stopping watcher * Apply suggestions from code review Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> * fix a deadlock when calling stop while emitting event is blocked * make sure to close the event channel after the event loop is done * add go doc * back date file instead of sleeping * Apply suggestions from code review Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> * check error Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	3 years ago
R.B. Boyer	80dfcb1bcd	raft: update to v1.3.5 (#12325 ) This includes closing some leadership transfer gaps and adding snapshot restore progress logging.	3 years ago
FFMMM	78264a8030	Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311 ) This commit syncs ENT changes to the OSS repo. Original commit details in ENT: ``` commit 569d25f7f4578981c3801e6e067295668210f748 Author: FFMMM <FFMMM@users.noreply.github.com> Date: Thu Feb 10 10:23:33 2022 -0800 Vendor fork net rpc (#1538) * replace net/rpc w consul-net-rpc/net/rpc Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> * replace msgpackrpc and go-msgpack with fork from mono repo Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> * gofmt all files touched Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> ``` Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>	3 years ago
Dhia Ayachi	25c36af222	update serf to v0.9.7 (#12057 ) * update serf to v0.9.7 * add change log * update changelog	3 years ago
dependabot[bot]	f3ac9dafa6	build(deps): bump github.com/ryanuber/columnize (#12062 ) Bumps [github.com/ryanuber/columnize](https://github.com/ryanuber/columnize) from 2.1.0+incompatible to 2.1.2+incompatible. - [Release notes](https://github.com/ryanuber/columnize/releases) - [Commits](https://github.com/ryanuber/columnize/compare/v2.1.0...v2.1.2) --- updated-dependencies: - dependency-name: github.com/ryanuber/columnize dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
dependabot[bot]	2c8de28005	build(deps): bump github.com/kr/text from 0.1.0 to 0.2.0 (#12063 ) Bumps [github.com/kr/text](https://github.com/kr/text) from 0.1.0 to 0.2.0. - [Release notes](https://github.com/kr/text/releases) - [Commits](https://github.com/kr/text/compare/v0.1.0...v0.2.0) --- updated-dependencies: - dependency-name: github.com/kr/text dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
dependabot[bot]	4947d6d29f	build(deps): bump github.com/mitchellh/pointerstructure (#12072 ) Bumps [github.com/mitchellh/pointerstructure](https://github.com/mitchellh/pointerstructure) from 1.0.0 to 1.2.1. - [Release notes](https://github.com/mitchellh/pointerstructure/releases) - [Commits](https://github.com/mitchellh/pointerstructure/compare/v1.0.0...v1.2.1) --- updated-dependencies: - dependency-name: github.com/mitchellh/pointerstructure dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
dependabot[bot]	97348e25a7	Bump github.com/aws/aws-sdk-go from 1.25.41 to 1.42.34 (#12083 ) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.25.41 to 1.42.34. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go/compare/v1.25.41...v1.42.34) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
dependabot[bot]	151e516a8c	Bump github.com/hashicorp/go-memdb from 1.3.1 to 1.3.2 (#11066 ) Bumps [github.com/hashicorp/go-memdb](https://github.com/hashicorp/go-memdb) from 1.3.1 to 1.3.2. - [Release notes](https://github.com/hashicorp/go-memdb/releases) - [Changelog](https://github.com/hashicorp/go-memdb/blob/master/changes.go) - [Commits](https://github.com/hashicorp/go-memdb/compare/v1.3.1...v1.3.2) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-memdb dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
dependabot[bot]	b9ed140b4b	Bump github.com/hashicorp/go-raftchunking from 0.6.1 to 0.6.2 (#11065 ) Bumps [github.com/hashicorp/go-raftchunking](https://github.com/hashicorp/go-raftchunking) from 0.6.1 to 0.6.2. - [Release notes](https://github.com/hashicorp/go-raftchunking/releases) - [Commits](https://github.com/hashicorp/go-raftchunking/compare/v0.6.1...v0.6.2) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-raftchunking dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
dependabot[bot]	8878f0ea21	build(deps): bump github.com/hashicorp/go-multierror from 1.1.0 to 1.1.1 Bumps [github.com/hashicorp/go-multierror](https://github.com/hashicorp/go-multierror) from 1.1.0 to 1.1.1. - [Release notes](https://github.com/hashicorp/go-multierror/releases) - [Commits](https://github.com/hashicorp/go-multierror/compare/v1.1.0...v1.1.1) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-multierror dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	3 years ago
Chris S. Kim	ea0df29afb	Update memberlist to 0.3.1 (#12042 )	3 years ago
Dhia Ayachi	d0274d38a9	upgrade raft to v1.3.3 (#11958 ) * upgrade raft to v1.3.3 * add change log * reword the changelog Co-authored-by: FFMMM <FFMMM@users.noreply.github.com> Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>	3 years ago
kisunji	59ddc44cc7	Update golang.org/x/net to address CVE-2021-44716	3 years ago
Matt Keeler	42a5635bc3	Use raft-boltdb/v2	3 years ago
Mike Morris	25826e3ee4	deps: update gopsutil to fix Windows ARM and macOS non-Apple LLVM builds (#11586 ) Bumps transitive dep go-ole to v1.2.6 with fixes	3 years ago
R.B. Boyer	eb21649f82	partitions: various refactors to support partitioning the serf LAN pool (#11568 )	3 years ago
FFMMM	0954d261ae	use *telemetry.MetricsPrefix as prometheus.PrometheusOpts.Name (#11290 ) Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>	3 years ago
Dhia Ayachi	ab31c50915	update raft to v1.3.2 (#11375 ) * update raft to v1.3.2 * add changelog * fix changelog	3 years ago
Giulio Micheloni	fecce25658	Separete test file and no stack trace in ret error	3 years ago
Jeff Widman	2dc62aa0c4	Bump `go-discover` to fix broken dep tree (#10898 )	3 years ago
freddygv	a78390a30b	Update yamux	3 years ago

16 Commits (NET-1805-cleanup-test-container-lib-basic-topology-single-dc)