75 Commits (f6424ab6a78fa76163d92785c213acd6e109df90)

Author	SHA1	Message	Date
Dan Upton	b117eb0126	resource: enforce consistent naming of resource types (#17611) ... For consistency, resource type names must follow these rules: - `Group` must be snake case, and in most cases a single word. - `GroupVersion` must be lowercase, start with a "v" and end with a number. - `Kind` must be pascal case. These were chosen because they map to our protobuf type naming conventions.	1 year ago
Dan Upton	48445dfa55	resource: add `AuthorizerContext` helper method (#17393)	1 year ago
Matt Keeler	37636eab71	Catalog V2 Container Based Integration Test (#17674) ... * Implement the Catalog V2 controller integration container tests This now allows the container tests to import things from the root module. However for now we want to be very restrictive about which packages we allow importing. * Add an upgrade test for the new catalog Currently this should be dormant and not executed. However its put in place to detect breaking changes in the future and show an example of how to do an upgrade test with integration tests structured like catalog v2. * Make testutil.Retry capable of performing cleanup operations These cleanup operations are executed after each retry attempt. * Move TestContext to taking an interface instead of a concrete testing.T This allows this to be used on a retry.R or generally anything that meets the interface. * Move to using TestContext instead of background contexts Also this forces all test methods to implement the Cleanup method now instead of that being an optional interface. Co-authored-by: Daniel Upton <daniel@floppy.co>	1 year ago
Matt Keeler	653a886689	Implement a Catalog Controllers Lifecycle Integration Test (#17435) ... * Implement a Catalog Controllers Lifecycle Integration Test * Prevent triggering the race detector. This allows defining some variables for protobuf constants and using those in comparisons. Without that, something internal in the fmt package ended up looking at the protobuf message size cache and triggering the race detector.	1 year ago
Matt Keeler	a5ba889034	Implement the service endpoints controller (#17216) ... * Add a ReplaceType dep mapper and move them into their own file * Implement the service endpoints controller * Implement a Catalog Controllers Integration Test	1 year ago
Matt Keeler	77f44fa878	Various bits of cleanup detected when using Go Workspaces (#17462) ... TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors. This commit: 1. Fixes envoy-library-references.sh to work again 2. Ensures we are pulling in go-control-plane@v0.11.0 everywhere (previously it was at that version in some modules and others were much older) 3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it. 4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field. 4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules 5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment. 6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible 7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner. 8. Updated linter rules to prevent usage of golang/protobuf 9. Updated a leader peering test to account for an extra colon in a grpc error message.	1 year ago
Dan Stough	d935c7b466	[OSS] gRPC Blocking Queries (#17426) ... * feat: initial grpc blocking queries * changelog and docs update	2 years ago
Matt Keeler	1d6a0c8f21	Add the workload health controller (#17215)	2 years ago
R.B. Boyer	21c6e0e8e6	fix two typos (#17389)	2 years ago
Matt Keeler	d37572bd44	Add a Node health controller (#17214) ... This will aggregate all HealthStatus objects owned by the Node and update the status of the Node with an overall health.	2 years ago
Dan Upton	879b775459	docs: initial documentation for the new State Store (#17315)	2 years ago
Matt Keeler	456156ebec	Add type validations for the catalog resources (#17211) ... Also adding some common resource validation error types to the internal/resource package.	2 years ago
Dan Upton	6c24a66f73	resource: optionally compare timestamps in `EqualStatus` (#17275)	2 years ago
Semir Patel	40eefaba18	Reaper controller for cascading deletes of owner resources (#17256)	2 years ago
Dan Upton	d53a1d4a27	resource: add helpers for more efficiently comparing IDs etc (#17224)	2 years ago
Dan Upton	972998203e	controller: deduplicate items in queue (#17168)	2 years ago
Dan Upton	6e1bc57469	Controller Runtime	2 years ago
Matt Keeler	34915670f2	Register new catalog & mesh protobuf types with the resource registry (#17225)	2 years ago
Semir Patel	991a002fcc	resource: List resources by owner (#17190)	2 years ago
Semir Patel	9fef1c7f17	Create tombstone on resource `Delete` (#17108)	2 years ago
Dan Upton	eeaa636164	Cleanup from unblocking the pipeline 🧹 (#17121)	2 years ago
Semir Patel	e7bb8fdf15	Fix or disable pipeline breaking changes that made it into main in last day or so (#17130) ... * Fix straggler from renaming Register->RegisterTypes * somehow a lint failure got through previously * Fix lint-consul-retry errors * adding in fix for success jobs getting skipped. (#17132) * Temporarily disable inmem backend conformance test to get green pipeline * Another test needs disabling --------- Co-authored-by: John Murret <john.murret@hashicorp.com>	2 years ago
Dan Upton	b9c485dcb8	Controller Supervision (#17016)	2 years ago
Dan Upton	ba4a314772	storage: fix bug where WatchList would (rarely) return duplicate events (#17067)	2 years ago
hashicorp-copywrite[bot]	9f81fc01e9	[COMPLIANCE] Add Copyright and License Headers (#16854) ... Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>	2 years ago
Semir Patel	2f7d591702	Tenancy wildcard validaton for `Write`, `Read`, and `Delete` endpoints (#17004)	2 years ago
Semir Patel	b8c9e133be	Add mutate hook to `Write` endpoint (#16958)	2 years ago
Semir Patel	3b83c7ee9a	Enforce ACLs on resource `Write` and `Delete` endpoints (#16956)	2 years ago
Dan Upton	d595e6ade9	resource: `WriteStatus` endpoint (#16886)	2 years ago
Semir Patel	317240fca7	Resource validation hook for `Write` endpoint (#16950)	2 years ago
Semir Patel	686f49346c	Check acls on resource `Read`, `List`, and `WatchList` (#16842)	2 years ago
Dan Upton	4fa2537b3b	Resource `Write` endpoint (#16786)	2 years ago
Dan Upton	671d5825ca	Raft storage backend (#16619)	2 years ago
Dan Upton	651549c97d	storage: fix resource leak in Watch (#16817)	2 years ago
Ronald	b64674623e	Copyright headers for missing files/folders (#16708) ... * copyright headers for agent folder	2 years ago
Dan Upton	81df781e5f	Add storage backend interface and in-memory implementation (#16538) ... Introduces `storage.Backend`, which will serve as the interface between the Resource Service and the underlying storage system (Raft today, but in the future, who knows!). The primary design goal of this interface is to keep its surface area small, and push as much functionality as possible into the layers above, so that new implementations can be added with little effort, and easily proven to be correct. To that end, we also provide a suite of "conformance" tests that can be run against a backend implementation to check it behaves correctly. In this commit, we introduce an initial in-memory storage backend, which is suitable for tests and when running Consul in development mode. This backend is a thin wrapper around the `Store` type, which implements a resource database using go-memdb and our internal pub/sub system. `Store` will also be used to handle reads in our Raft backend, and in the future, used as a local cache for external storage systems.	2 years ago
Poonam Jadhav	9c64731a56	feat: add category annotation to RPC and gRPC methods (#16646)	2 years ago
Semir Patel	5a3fec6238	Basic resource type registry (#16622)	2 years ago
Matt Keeler	84156afe87	Remove private prefix from proto-gen-rpc-glue e2e test (#16433)	2 years ago
Matt Keeler	085c0addc0	Protobuf Refactoring for Multi-Module Cleanliness (#16302) ... Protobuf Refactoring for Multi-Module Cleanliness This commit includes the following: Moves all packages that were within proto/ to proto/private Rewrites imports to account for the packages being moved Adds in buf.work.yaml to enable buf workspaces Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes) Why: In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage. There were some recent changes to have our own ratelimiting annotations. The two combined were not working when I was trying to use them together (attempting to rebase another branch) Buf workspaces should be the solution to the problem Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root. This resulted in proto file name conflicts in the Go global protobuf type registry. The solution to that was to add in a private/ directory into the path within the proto/ directory. That then required rewriting all the imports. Is this safe? AFAICT yes The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc) Other than imports, there were no changes to any generated code as a result of this.	2 years ago
Matt Keeler	5afd4657ec	Protobuf Modernization (#15949) ... * Protobuf Modernization Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf Marshallers (protobuf and json) needed some changes to account for different APIs. Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package. This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces. * Fix go-mod-tidy make target to work on all modules	2 years ago
Dan Upton	ee2d47da83	Fix missing comment in generated enterprise rate-limit file (#15895) ... Fixes bug in #15564 where gofmt would strip out the generated code warning comment because it was on the same line as the build tag.	2 years ago
Dan Upton	d4c435856b	grpc: `protoc` plugin for generating gRPC rate limit specifications (#15564) ... Adds automation for generating the map of `gRPC Method Name → Rate Limit Type` used by the middleware introduced in #15550, and will ensure we don't forget to add new endpoints. Engineers must annotate their RPCs in the proto file like so: ``` rpc Foo(FooRequest) returns (FooResponse) { option (consul.internal.ratelimit.spec) = { operation_type: READ, }; } ``` When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will be installed that writes rate-limit specs as a JSON array to a file called `.ratelimit.tmp` (one per protobuf package/directory). After running Buf, `make proto` will execute a post-process script that will ingest all of the `.ratelimit.tmp` files and generate a Go file containing the mappings in the `agent/grpc-middleware` package. In the enterprise repository, it will write an additional file with the enterprise-only endpoints. If an engineer forgets to add the annotation to a new RPC, the plugin will return an error like so: ``` RPC Foo is missing rate-limit specification, fix it with: import "proto-public/annotations/ratelimit/ratelimit.proto"; service Bar { rpc Foo(...) returns (...) { option (hashicorp.consul.internal.ratelimit.spec) = { operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT, }; } } ``` In the future, this annotation can be extended to support rate-limit category (e.g. KV vs Catalog) and to determine the retry policy.	2 years ago
Chris S. Kim	cc819ad83b	[OSS] Add boilerplate for proto files implementing BlockableQuery (#15554)	2 years ago
Kyle Schochenmaier	bf0f61a878	removes ioutil usage everywhere which was deprecated in go1.16 (#15297) ... * update go version to 1.18 for api and sdk, go mod tidy * removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward. Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	2 years ago
Chris S. Kim	bde57c0dd0	Regenerate files according to 1.19.2 formatter	2 years ago
Chris S. Kim	3d05a8e9af	Remove unused methods from template	2 years ago
Chris S. Kim	29a297d3e9	Refactor client RPC timeouts (#14965) ... Fix an issue where rpc_hold_timeout was being used as the timeout for non-blocking queries. Users should be able to tune read timeouts without fiddling with rpc_hold_timeout. A new configuration `rpc_read_timeout` is created. Refactor some implementation from the original PR 11500 to remove the misleading linkage between RPCInfo's timeout (used to retry in case of certain modes of failures) and the client RPC timeouts.	2 years ago
Paul Glass	77afe0e76e	Extract AWS auth implementation out of Consul (#13760)	2 years ago
alex	7c0daeade8	fix leader annotation (#13786) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago