Commit Graph

206 Commits (a152e0ee7c440db8459c79e83af0a16f2cb1b007)

Author SHA1 Message Date
Hans Hasselberg b6097a99b8
Add new fields to HCP bootstrap config request and push state request
To support linking cluster, HCP needs to know the datacenter and if ACLs are enabled. Otherwise hosted Consul Core UI won't work properly.
2023-05-12 21:01:56 -06:00
Freddy e02ef16f02
Update HCP bootstrapping to support existing clusters (#16916)
* Persist HCP management token from server config

We want to move away from injecting an initial management token into
Consul clusters linked to HCP. The reasoning is that by using a separate
class of token we can have more flexibility in terms of allowing HCP's
token to co-exist with the user's management token.

Down the line we can also more easily adjust the permissions attached to
HCP's token to limit it's scope.

With these changes, the cloud management token is like the initial
management token in that iit has the same global management policy and
if it is created it effectively bootstraps the ACL system.

* Update SDK and mock HCP server

The HCP management token will now be sent in a special field rather than
as Consul's "initial management" token configuration.

This commit also updates the mock HCP server to more accurately reflect
the behavior of the CCM backend.

* Refactor HCP bootstrapping logic and add tests

We want to allow users to link Consul clusters that already exist to
HCP. Existing clusters need care when bootstrapped by HCP, since we do
not want to do things like change ACL/TLS settings for a running
cluster.

Additional changes:

* Deconstruct MaybeBootstrap so that it can be tested. The HCP Go SDK
  requires HTTPS to fetch a token from the Auth URL, even if the backend
  server is mocked. By pulling the hcp.Client creation out we can modify
  its TLS configuration in tests while keeping the secure behavior in
  production code.

* Add light validation for data received/loaded.

* Sanitize initial_management token from received config, since HCP will
  only ever use the CloudConfig.MangementToken.

* Add changelog entry
2023-04-27 22:27:39 +02:00
John Maguire e47f3216e5
APIGW Normalize Status Conditions (#16994)
* normalize status conditions for gateways and routes

* Added tests for checking condition status and panic conditions for
validating combinations, added dummy code for fsm store

* get rid of unneeded gateway condition generator struct

* Remove unused file

* run go mod tidy

* Update tests, add conflicted gateway status

* put back removed status for test

* Fix linting violation, remove custom conflicted status

* Update fsm commands oss

* Fix incorrect combination of type/condition/status

* cleaning up from PR review

* Change "invalidCertificate" to be of accepted status

* Move status condition enums into api package

* Update gateways controller and generated code

* Update conditions in fsm oss tests

* run go mod tidy on consul-container module to fix linting

* Fix type for gateway endpoint test

* go mod tidy from changes to api

* go mod tidy on troubleshoot

* Fix route conflicted reason

* fix route conflict reason rename

* Fix text for gateway conflicted status

* Add valid certificate ref condition setting

* Revert change to resolved refs to be handled in future PR
2023-04-24 16:22:55 -04:00
Paul Banks a011d8c944
Bump raft to 1.5.0 (#17081)
* Bump raft to 1.5.0

* Add CHANGELOG entry

* Add CHANGELOG entry with right extension (thanks VSCode)

* Add CHANGELOG entry with right extension (thanks VSCode)

* Go mod tidy
2023-04-21 20:13:55 +01:00
Kevin Wang 268f93e6f4
Bump the golang.org/x/net to 0.7.0 to address CVE-2022-41723 (#16754)
* Bump the golang.org/x/net to 0.7.0 to address CVE-2022-41723

https://nvd.nist.gov/vuln/detail/CVE-2022-41723

* Add changelog entry

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-04-18 17:31:08 +00:00
Dan Upton 4fa2537b3b
Resource `Write` endpoint (#16786) 2023-04-06 10:40:04 +01:00
Dhia Ayachi b9d8552e25
Snapshot restore tests (#16647)
* add snapshot restore test

* add logstore as test parameter

* Use the correct image version

* make sure we read the logs from a followers to test the follower snapshot install path.

* update to raf-wal v0.3.0

* add changelog.

* updating changelog for bug description and removed integration test.

* setting up test container builder to only set logStore for 1.15 and higher

---------

Co-authored-by: Paul Banks <pbanks@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-03-18 14:43:22 -06:00
Semir Patel 9a5cb20570
Bump submodules from latest 1.15.1 patch release (#16578)
* Update changelog with Consul patch releases 1.13.7, 1.14.5, 1.15.1

* Bump submodules from latest patch release

* Forgot one
2023-03-08 14:37:50 -06:00
John Eikenberry f5641ffccc
support vault auth config for alicloud ca provider
Add support for using existing vault auto-auth configurations as the
provider configuration when using Vault's CA provider with AliCloud.

AliCloud requires 2 extra fields to enable it to use STS (it's preferred
auth setup). Our vault-plugin-auth-alicloud package contained a method
to help generate them as they require you to make an http call to
a faked endpoint proxy to get them (url and headers base64 encoded).
2023-03-07 03:02:05 +00:00
Dan Stough f1436109ea
[OSS] security: update go to 1.20.1 (#16263)
* security: update go to 1.20.1
2023-02-17 15:04:12 -05:00
Semir Patel 8979e64a94
Bump x/time to 0.3.0 and fix related breakage linked to RPCRateLimit (#16241)
* Bump x/time to 0.3.0 and fix related breakage linked to RPCRateLimit initialization

* Apply limitVal(...) to other rate.Limit config fields
2023-02-13 11:11:51 -06:00
Nitya Dhanushkodi 80fb18aa35
troubleshoot: make output have tables and colors (#16235)
Adds tables and colors using libraries used in consul-k8s. It doesn't add the full `terminal` UI package that consul-k8s uses since there is an existing UI in Consul that I didn't want to affect too much. So instead this adds to the existing UI.
2023-02-10 11:12:13 -08:00
Nitya Dhanushkodi 78a4b5fd6f
update the api in envoyextensions and troubleshoot modules (#16226) 2023-02-09 15:39:03 -08:00
Paul Banks 5397e9ee7f
Adding experimental support for a more efficient LogStore implementation (#16176)
* Adding experimental support for a more efficient LogStore implementation

* Adding changelog entry

* Fix go mod tidy issues
2023-02-08 16:50:22 +00:00
cskh e91bc9c058
feat: envoy extension - http local rate limit (#16196)
- http local rate limit
- Apply rate limit only to local_app
- unit test and integ test
2023-02-07 21:56:15 -05:00
dependabot[bot] 256320babf
Bump github.com/prometheus/client_golang from 1.4.0 to 1.14.0 (#15292)
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.4.0 to 1.14.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.4.0...v1.14.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2023-02-07 17:32:31 -06:00
Nitya Dhanushkodi b8b37c2357
refactor: remove troubleshoot module dependency on consul top level module (#16162)
Ensure nothing in the troubleshoot go module depends on consul's top level module. This is so we can import troubleshoot into consul-k8s and not import all of consul.

* turns troubleshoot into a go module [authored by @curtbushko]
* gets the envoy protos into the troubleshoot module [authored by @curtbushko]
* adds a new go module `envoyextensions` which has xdscommon and extensioncommon folders that both the xds package and the troubleshoot package can import
* adds testing and linting for the new go modules
* moves the unit tests in `troubleshoot/validateupstream` that depend on proxycfg/xds into the xds package, with a comment describing why those tests cannot be in the troubleshoot package
* fixes all the imports everywhere as a result of these changes 

Co-authored-by: Curt Bushko <cbushko@gmail.com>
2023-02-06 09:14:35 -08:00
Chris Thain 2f4c8e50f2
Support Vault agent auth config for AWS/GCP CA provider auth (#15970) 2023-01-18 11:53:04 -08:00
Frank DiRocco 59a3a0749c
Update go-discover to support ECS discovery (#13782)
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-01-12 12:06:29 -06:00
Dan Stough 6d2880e894
feat: add access logs to dataplane bootstrap rpc (#15951) 2023-01-11 13:40:09 -05:00
Matt Keeler 5afd4657ec
Protobuf Modernization (#15949)
* Protobuf Modernization

Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf

Marshallers (protobuf and json) needed some changes to account for different APIs.

Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package.

This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces.

* Fix go-mod-tidy make target to work on all modules
2023-01-11 09:39:10 -05:00
Semir Patel bafa5c7156
Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
Dan Stough 98ef5f28dd
[OSS] security: update x/net module (#15737)
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-12-08 16:45:44 -05:00
Dhia Ayachi 81e40c1fac
add multilimiter and tests (#15467)
* add multilimiter and tests

* exporting LimitedEntity

* go mod tidy

* Apply suggestions from code review

Co-authored-by: John Murret <john.murret@hashicorp.com>

* add config update and rename config params

* add doc string and split config

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* use timer to avoid go routine leak and change the interface

* add comments to tests

* fix failing test

* add prefix with config edge, refactor tests

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* refactor to apply configs for limiters under a prefix

* add fuzz tests and fix bugs found. Refactor reconcile loop to have a simpler logic

* make KeyType an exported type

* split the config and limiter trees to fix race conditions in config update

* rename variables

* fix race in test and remove dead code

* fix reconcile loop to not create a timer on each loop

* add extra benchmark tests and fix tests

* fix benchmark test to pass value to func

* use a separate go routine to write limiters (#15643)

* use a separate go routine to write limiters

* Add updating limiter when another limiter is created

* fix waiter to be a ticker, so we commit more than once.

* fix tests and add tests for coverage

* unexport members and add tests

* make UpdateConfig thread safe and multi call to Run safe

* replace swith with if

* fix review comments

* replace time.sleep with retries

* fix flaky test and remove unnecessary init

* fix test races

* remove unnecessary negative test case

* remove fixed todo

Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
2022-12-08 14:42:07 -05:00
Chris S. Kim 10349bd84b
clean up go.mod (#15638) 2022-12-01 16:24:35 +00:00
Dan Stough 227fd14287
chore: updates from 1.14.2 release (#15633)
* chore: updates from 1.14.2 release
2022-11-30 22:15:58 -05:00
Derek Menteer 8079686bf0
Add 1.14.1 release updates. (#15514)
Add post-release changes for 1.14.1 updates.
2022-11-21 13:35:30 -06:00
Derek Menteer dc27e35f82
Consul 1.14 post-release updates (#15382)
* Update changelog with 1.14 notes.

* gomod version bumps for 1.14 release.
2022-11-15 14:22:43 -06:00
Chris S. Kim 985a4ee1b1
Update hcp-scada-provider to fix diamond dependency problem with go-msgpack (#15185) 2022-11-07 11:34:30 -05:00
Chris S. Kim 41019adf56 Update go version to 1.19 2022-10-24 16:12:08 -04:00
Derek Menteer 3f2403ffe4 Update go mod file for api package. 2022-10-20 15:08:03 -05:00
Chris S. Kim 59f14f5e01 Update go-memdb to fix a goroutine leak 2022-10-20 14:32:42 -04:00
Dhia Ayachi 5eb2bad8fd
bump relevant modules versions (#14972) 2022-10-18 11:24:26 -04:00
Dhia Ayachi 7f6c52a9ee
bump raft version to v1.3.11 (#14897)
* bump raft version to v1.3.11

* Add change log

* fix go.sum
2022-10-12 08:51:52 -04:00
John Murret 79a541fd7d
Upgrade serf to v0.10.1 and memberlist to v0.5.0 to get memberlist size metrics and broadcast queue depth metric (#14873)
* updating to serf v0.10.1 and memberlist v0.5.0 to get memberlist size metrics and memberlist broadcast queue depth metric

* update changelog

* update changelog

* correcting changelog

* adding "QueueCheckInterval" for memberlist to test

* updating integration test containers to grab latest api
2022-10-04 17:51:37 -06:00
Nick Ethier 1c1b0994b8
add HCP integration component (#14723)
* add HCP integration

* lint: use non-deprecated logging interface
2022-09-26 14:58:15 -04:00
DanStough 7704daaad5 release updates for 1.13.2, 1.12.5, and 1.11.9 2022-09-21 15:07:44 -04:00
freddygv f30bc96239 Test fixes
- Pulls in CLI test fix from main
- Updates psutils to fix TestAgent_Host on M1 Mac
2022-09-16 17:57:10 -06:00
Daniel Upton 939a1ae34d Make proto-public a Go module
Our original intention was for projects to consume and generate their
own Go code for these protobuf packages using Buf. While this is still
the best route for many projects, it causes some headaches when using
a library (e.g. consul-server-connection-manager) that pulls in the
same protobuf package as your project, as Go's protobuf implementation
only allows for a package/namespace to be registered once.

In such cases, projects can depend on this Go module instead, as a
single place where these protobuf packages are registered.
2022-09-06 19:30:17 +01:00
cskh e84e4b8868
Fix: upgrade pkg imdario/merg to prevent merge config panic (#14237)
* upgrade imdario/merg to prevent merge config panic

* test: service definition takes precedence over service-defaults in merged results
2022-08-17 21:14:04 -04:00
cskh 81931e52c3
feat(telemetry): add labels to serf and memberlist metrics (#14161)
* feat(telemetry): add labels to serf and memberlist metrics
* changelog
* doc update

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2022-08-11 22:09:56 -04:00
Paul Glass 77afe0e76e
Extract AWS auth implementation out of Consul (#13760) 2022-07-19 16:26:44 -05:00
Evan Culver d523d005d9
Latest submodule versions (#13750) 2022-07-15 09:58:21 -07:00
Daniel Kimsey feead0b11b Update go-grpc/grpc to resolve conection memory leak
Reported in #12288

The initial test reported was ported and accurately reproduced the issue.
However, since it is a test of an upstream library's internal behavior it won't
be codified in our test suite. Refer to the ticket/PR for details on how to
demonstrate the behavior.
2022-06-08 11:29:29 +01:00
Chris S. Kim f0a9b30174
Update repo to use go:embed (#10996)
Replace bindata packages with stdlib go:embed.
Modernize some uiserver code with newer interfaces introduced in go 1.16 (mainly working with fs.File instead of http.File.
Remove steps that are no longer used from our build files.
Add Github Action to detect differences in agent/uiserver/dist and verify that the files are correct (by compiling UI assets and comparing contents).
2022-05-31 15:33:56 -04:00
Kyle Havlovitz 4bc6c23357 Add connection limit setting to service defaults 2022-05-24 10:13:38 -07:00
R.B. Boyer 2e72f44fda
peering: accept replication stream of discovery chain information at the importing side (#13151) 2022-05-19 16:37:52 -05:00
Dhia Ayachi 78412ae069
upgrade serf to v0.9.8 (#13062)
* upgrade serf to v0.9.8

* add changelog

* Update .changelog/13062.txt

Co-authored-by: Dan Upton <daniel@floppy.co>

Co-authored-by: Dan Upton <daniel@floppy.co>
2022-05-16 14:13:23 -04:00
R.B. Boyer b739ebc735
remove replace directive for istio.io/gogo-genproto (#12990)
This was added in #7194 and is no longer a transitive dependency.
2022-05-10 11:33:59 -05:00
Dhia Ayachi b895dd7d2d
change mod go version to 1.18 (#12976)
* change mod go version to 1.18

* fix go.mod format for 1.18
2022-05-09 13:29:43 -04:00