TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors.
This commit:
1. Fixes envoy-library-references.sh to work again
2. Ensures we are pulling in go-control-plane@v0.11.0 everywhere (previously it was at that version in some modules and others were much older)
3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it.
4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field.
4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules
5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment.
6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible
7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner.
8. Updated linter rules to prevent usage of golang/protobuf
9. Updated a leader peering test to account for an extra colon in a grpc error message.
* normalize status conditions for gateways and routes
* Added tests for checking condition status and panic conditions for
validating combinations, added dummy code for fsm store
* get rid of unneeded gateway condition generator struct
* Remove unused file
* run go mod tidy
* Update tests, add conflicted gateway status
* put back removed status for test
* Fix linting violation, remove custom conflicted status
* Update fsm commands oss
* Fix incorrect combination of type/condition/status
* cleaning up from PR review
* Change "invalidCertificate" to be of accepted status
* Move status condition enums into api package
* Update gateways controller and generated code
* Update conditions in fsm oss tests
* run go mod tidy on consul-container module to fix linting
* Fix type for gateway endpoint test
* go mod tidy from changes to api
* go mod tidy on troubleshoot
* Fix route conflicted reason
* fix route conflict reason rename
* Fix text for gateway conflicted status
* Add valid certificate ref condition setting
* Revert change to resolved refs to be handled in future PR
1. Upgraded agent can inherit the persisted token and join the cluster
2. Agent token prior to upgrade is still valid after upgrade
3. Enable ACL in the agent configuration
- remove dep on consul main module
- use 'consul tls' subcommands instead of tlsutil
- use direct json config construction instead of agent/config structs
- merge libcluster and libagent packages together
- more widely use BuildContext
- get the OSS/ENT runner stuff working properly
- reduce some flakiness
- fix some correctness related to http/https API
* Protobuf Modernization
Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf
Marshallers (protobuf and json) needed some changes to account for different APIs.
Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package.
This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces.
* Fix go-mod-tidy make target to work on all modules
Adds automation for generating the map of `gRPC Method Name → Rate Limit Type`
used by the middleware introduced in #15550, and will ensure we don't forget
to add new endpoints.
Engineers must annotate their RPCs in the proto file like so:
```
rpc Foo(FooRequest) returns (FooResponse) {
option (consul.internal.ratelimit.spec) = {
operation_type: READ,
};
}
```
When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will
be installed that writes rate-limit specs as a JSON array to a file called
`.ratelimit.tmp` (one per protobuf package/directory).
After running Buf, `make proto` will execute a post-process script that will
ingest all of the `.ratelimit.tmp` files and generate a Go file containing the
mappings in the `agent/grpc-middleware` package. In the enterprise repository,
it will write an additional file with the enterprise-only endpoints.
If an engineer forgets to add the annotation to a new RPC, the plugin will
return an error like so:
```
RPC Foo is missing rate-limit specification, fix it with:
import "proto-public/annotations/ratelimit/ratelimit.proto";
service Bar {
rpc Foo(...) returns (...) {
option (hashicorp.consul.internal.ratelimit.spec) = {
operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT,
};
}
}
```
In the future, this annotation can be extended to support rate-limit
category (e.g. KV vs Catalog) and to determine the retry policy.
* integ-test: test consul upgrade from the snapshot of a running cluster
* use Target version as default
Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
* updating to serf v0.10.1 and memberlist v0.5.0 to get memberlist size metrics and memberlist broadcast queue depth metric
* update changelog
* update changelog
* correcting changelog
* adding "QueueCheckInterval" for memberlist to test
* updating integration test containers to grab latest api
* add a sample
* Consul cluster test
* add build dockerfile
* add tests to cover mixed versions tests
* use flag to pass docker image name
* remove default config and rely on flags to inject the right image to test
* add cluster abstraction
* fix imports and remove old files
* fix imports and remove old files
* fix dockerIgnore
* make a `Node interface` and encapsulate ConsulContainer
* fix a test bug where we only check the leader against a single node.
* add upgrade tests to CI
* fix yaml alignment
* fix alignment take 2
* fix flag naming
* fix image to build
* fix test run and go mod tidy
* add a debug command
* run without RYUK
* fix parallel run
* add skip reaper code
* make tempdir in local dir
* chmod the temp dir to 0777
* chmod the right dir name
* change executor to use machine instead of docker
* add docker layer caching
* remove setup docker
* add gotestsum
* install go version
* use variable for GO installed version
* add environment
* add environment in the right place
* do not disable RYUK in CI
* add service check to tests
* assertions outside routines
* add queryBackend to the api query meta.
* check if we are using the right backend for those tests (streaming)
* change the tested endpoint to use one that have streaming.
* refactor to test multiple scenarios for streaming
* Fix dockerfile
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
* rename Clients to clients
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
* check if cluster have 0 node
* tidy code and add some doc strings
* use uuid instead of random string
* add doc strings to tests
* add queryBackend to the api query meta.
* add a changelog
* fix for api backend query
* add missing require
* fix q.QueryBackend
* Revert "fix q.QueryBackend"
This reverts commit cd0e5f7b1a.
* fix circle ci config
* tidy go mod after merging main
* rename package and fix test scenario
* update go download url
* address review comments
* rename flag in CI
* add readme to the upgrade tests
* fix golang download url
* fix golang arch downloaded
* fix AddNodes to handle an empty cluster case
* use `parseBool`
* rename circle job and add comment
* update testcontainer to 0.13
* fix circle ci config
* remove build docker file and use `make dev-docker` instead
* Apply suggestions from code review
Co-authored-by: Dan Upton <daniel@floppy.co>
* fix a typo
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
Matt Keeler