355 Commits (fea6926de39ed0615d73b7671288fe1af11493db)

Author	SHA1	Message	Date
Derek Menteer	eb7e20307c	[NET-5589] Add jitter to xds v2 leaf cert watches (#18940) ... Add jitter to xds v2 leaf cert watches.	1 year ago
Semir Patel	d2be7577b9	tenancy: split up tenancy `types.go` into CE version (#18966)	1 year ago
Matt Keeler	53fcc5d9a5	Add protoc generator to emit resource type variables (#18957) ... The annotations include a little more data than is strictly necessary because we will also have a protoc generator for openapi output.	1 year ago
Eric Haberkorn	f87ae3636c	Fix V2 Wildcard RBAC Regular Expressions (#18941) ... fix wildcard rbac regular expressions	1 year ago
Derek Menteer	d4ed3047f8	[NET-5589] Optimize leaf watch diff on xds controller. (#18921) ... Optimize leaf watch diff on xds controller.	1 year ago
John Murret	700d1bb37c	NET-5131 - support multiple ported upstreams tests (#18923) ... * add multiple upstream ports to golden file test for destination builder * NET-5131 - add unit tests for multiple ported upstreams * fix merge conflicts	1 year ago
Dhia Ayachi	341dc28ff9	Add namespace proto and registration (#18848) ... * add namespace proto and registration * fix proto generation * add missing copywrite headers * fix proto linter errors * fix exports and Type export * add mutate hook and more validation * add more validation rules and tests * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * fix owner error and add test * remove ACL for now * add tests around space suffix prefix. * only fait when ns and ap are default, add test for it --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com>	1 year ago
R.B. Boyer	d574473fd1	mesh: make FailoverPolicy work in xdsv2 and ProxyStateTemplate (#18900) ... Ensure that configuring a FailoverPolicy for a service that is reachable via a xRoute or a direct upstream causes an envoy aggregate cluster to be created for the original cluster name, but with separate clusters for each one of the possible destinations.	1 year ago
Nitya Dhanushkodi	3a2e62053a	v2: various fixes to make K8s tproxy multiport acceptance tests and manual explicit upstreams (single port) tests pass (#18874) ... Adding coauthors who mobbed/paired at various points throughout last week. Co-authored-by: Dan Stough <dan.stough@hashicorp.com> Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: John Murret <john.murret@hashicorp.com> Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: Michael Wilkerson <mwilkerson@hashicorp.com>	1 year ago
R.B. Boyer	07d916e84f	resource: ensure resource.AuthorizerContext properly strips the local… (#18908) ... resource: ensure resource.AuthorizerContext properly strips the local peer name	1 year ago
Eric Haberkorn	170417ac97	Honor Default Traffic Permissions in V2 (#18886) ... wire up v2 default traffic permissions	1 year ago
Iryna Shustava	212793a4ee	mesh: only build tproxy outbound listener once per destination (#18836) ... Previously, when using implicit upstreams, we'd build outbound listener per destination instead of one for all destinations. This will result in port conflicts when trying to send this config to envoy. This PR also makes sure that leaf and root references are always added (before we would only add it if there are inbound non-mesh ports). Also, black-hole traffic when there are no inbound ports other than mesh	1 year ago
Semir Patel	62796a1454	resource: mutate and validate before acls on write (#18868)	1 year ago
R.B. Boyer	dabbc9627b	mesh: normalize/default/validate tenancy components of mesh internal References (#18827) ... HTTPRoute, GRPCRoute, TCPRoute, and Upstreams resources contain inner Reference fields. We want to ensure that components of those reference Tenancy fields left unspecified are defaulted using the tenancy of the enclosing resource. As the underlying helper being used to do the normalization calls the function modified in #18822, it also means that the PeerName field will be set to "local" for now automatically to avoid "local" != "" issues downstream.	1 year ago
R.B. Boyer	696aa1bbd2	mesh: update xds controller to synthesize empty endpoints when no endpoints ref is found (#18835)	1 year ago
R.B. Boyer	b4d5178e5c	catalog: normalize/default/validate tenancy components of FailoverPolicy internal References (#18825) ... FailoverPolicy resources contain inner Reference fields. We want to ensure that components of those reference Tenancy fields left unspecified are defaulted using the tenancy of the enclosing FailoverPolicy resource. As the underlying helper being used to do the normalization calls the function modified in #18822, it also means that the PeerName field will be set to "local" for now automatically to avoid "local" != "" issues downstream.	1 year ago
Dhia Ayachi	4435e4a420	add v2 tenancy bridge Flag and v2 Tenancy Bridge initial implementation (#18830) ... * add v2 tenancy bridge and a feature flag for v2 tenancy * move tenancy bridge v2 under resource package	1 year ago
Iryna Shustava	a89938e0c1	catalog: Default protocol to tcp in catalog.Service if unspecified (#18832)	1 year ago
R.B. Boyer	5cde50dee7	mesh: prevent writing a ComputedRoutes with no ported configs (#18833)	1 year ago
skpratt	1fda2965e8	Allow empty data writes for resources (#18819) ... * allow nil data writes for resources * update demo to test valid type with no data	1 year ago
Iryna Shustava	6838441c54	Default to tcp protocol when workload protocol is unspecified (#18824)	1 year ago
Eric Haberkorn	21fdbbabbc	Wire up traffic permissions (#18812) ... Wire up traffic permissions	1 year ago
Semir Patel	d3dad14030	resource: default peername to "local" for now (#18822)	1 year ago
R.B. Boyer	9c1a1ffcde	mesh: call the right mapper (#18818)	1 year ago
R.B. Boyer	66e1cdf40c	mesh: Wire ComputedRoutes into the ProxyStateTemplate via the sidecar controller (#18752) ... Reworks the sidecar controller to accept ComputedRoutes as an input and use it to generate appropriate ProxyStateTemplate resources containing L4/L7 mesh configuration.	1 year ago
skpratt	e5808d85f7	register traffic permission and workload identity types (#18704) ... * add workload identity and traffic permission protos * register new types * add generated pb code * fix exports.go path * add proto newlines * fix type name Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com> * address review * fix protos and add tests * fix validation constraints * add tests --------- Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>	1 year ago
Dhia Ayachi	658c27a684	add fuzz tests to resourcehcl package and fix some panics (#18798) ... * add a simple fuzz test for the resourcehcl package and fix some panics uncovered by the test * fix default for null values	1 year ago
Iryna Shustava	7f2a1d9812	catalog: service endpoints inherits protocol from service when workload doesn't have one (#18792)	1 year ago
R.B. Boyer	07f54fe3b8	resource: add helper to normalize inner Reference tenancy during mutate (#18765) ... When one resource contains an inner field that is of type *pbresource.Reference we want the Tenancy to be reasonably defaulted by the following rules: 1. The final values will be limited by the scope of the referenced type. 2. Values will be inferred from the parent's tenancy, and if that is insufficient then using the default tenancy for the type's scope. 3. Namespace will only be used from a parent if the reference and the parent share a partition, otherwise the default namespace will be used. Until we tackle peering, this hard codes an assumption of peer name being local. The logic for defaulting may need adjustment when that is addressed.	1 year ago
Eric Haberkorn	12be06f8e5	Add V2 TCP traffic permissions (#18771) ... Add support for TCP traffic permissions	1 year ago
Nitya Dhanushkodi	78b170ad50	xds controller: setup watches for and compute leaf cert references in ProxyStateTemplate, and wire up leaf cert manager dependency (#18756) ... * Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager. The leaf cert logic in the controller: * Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too). * Gets the leaf cert from the leaf cert cache * Stores the leaf cert in the ProxyState that's pushed to xds * For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource. The wrapper allows mapping events to resources (as opposed to mapping resources to resources) The controller tests: Unit: Ensure that we resolve leaf cert references Lifecycle: Ensure that when the CA is updated, the leaf cert is as well Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id. * Pulls out some leaf cert test helpers into a helpers file so it can be used in the xds controller tests. * Wires up leaf cert manager dependency * Support getting token from proxytracker * Add workload identity spiffe id type to the authorize and sign functions --------- Co-authored-by: John Murret <john.murret@hashicorp.com>	1 year ago
R.B. Boyer	89e6725eee	mesh: create new routes-controller to reconcile xRoute types into a ComputedRoutes resource (#18460) ... This new controller produces an intermediate output (ComputedRoutes) that is meant to summarize all relevant xRoutes and related mesh configuration in an easier-to-use format for downstream use to construct the ProxyStateTemplate. It also applies status updates to the xRoute resource types to indicate that they are themselves semantically valid inputs.	1 year ago
Poonam Jadhav	264166fcc0	fix: write endpoint errors out gracefully (#18743)	1 year ago
John Murret	62062fd4fd	NET-5132 - Configure multiport routing for connect proxies in TProxy mode (#18606) ... * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Address PR comments * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * PR review comments * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Make sure endpoint refs route to mesh port instead of an app port * Address PR comments * fixing copyright * tidy imports * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * tidy imports * add copyright headers * Prefix sidecar proxy test files with source and destination. * Update controller_test.go * NET-5132 - Configure multiport routing for connect proxies in TProxy mode * formatting golden files * reverting golden files and adding changes in manually. build implicit destinations still has some issues. * fixing files that were incorrectly repeating the outbound listener * PR comments * extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName) * removing address level filtering. * adding license to resources_test.go --------- Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>	1 year ago
Dhia Ayachi	b1688ad856	Run copyright after running deep-copy as part of the Makefile/CI (#18741) ... * execute copyright headers after performing deep-copy generation. * fix copyright install * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Apply suggestions from code review Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * rename steps to match codegen naming * remove copywrite install category --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com>	1 year ago
R.B. Boyer	a69e901660	xds: update golden tests to be deterministic (#18707)	1 year ago
John Murret	3e78b4cf34	Prefix sidecar proxy test files with source and destination. (#18620) ... * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Address PR comments * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * PR review comments * mesh-controller: handle L4 protocols for a proxy without upstreams * sidecar-controller: Support explicit destinations for L4 protocols and single ports. * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * endpoints-controller: add workload identity to the service endpoints resource * small fixes * review comments * Make sure endpoint refs route to mesh port instead of an app port * Address PR comments * fixing copyright * tidy imports * sidecar-proxy controller: Add support for transparent proxy This currently does not support inferring destinations from intentions. * tidy imports * add copyright headers * Prefix sidecar proxy test files with source and destination. * Update controller_test.go --------- Co-authored-by: Iryna Shustava <iryna@hashicorp.com> Co-authored-by: R.B. Boyer <rb@hashicorp.com> Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>	1 year ago
Iryna Shustava	1557e1d6a3	sidecar-proxy controller: Add support for transparent proxy (NET-5069) (#18458) ... This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations.	1 year ago
R.B. Boyer	acd9b3d1c4	test: update sidecarproxy/builder golden tests to use determinstic golden data (#18703)	1 year ago
Iryna Shustava	3c70e14713	sidecar-proxy controller: L4 controller with explicit upstreams (NET-3988) (#18352) ... * This controller generates and saves ProxyStateTemplate for sidecar proxies. * It currently supports single-port L4 ports only. * It keeps a cache of all destinations to make it easier to compute and retrieve destinations. * It will update the status of the pbmesh.Upstreams resource if anything is invalid. * This commit also changes service endpoints to include workload identity. This made the implementation a bit easier as we don't need to look up as many workloads and instead rely on endpoints data.	1 year ago
wangxinyi7	df9d12a56a	Net 2714/xw cli read command (#18462) ... enable `consul resource read` command in cli	1 year ago
Semir Patel	b96cff7436	resource: Require scope for resource registration (#18635)	1 year ago
Michael Zalimeni	699aa47416	fix: make UNSPECIFIED protocol pass validation (#18634) ... We explicitly enumerate the allowed protocols in validation, so this change is necessary to use the new enum value. Also add tests for enum validators to ensure they stay aligned to protos unless we explicitly want them to diverge.	1 year ago
Dhia Ayachi	f8d77f027a	delete all v2 resources type when deleting a namespace (CE) (#18621) ... * add namespace scope to ServiceV1Alpha1Type * add CE portion of namespace deletion	1 year ago
Ashwin Venkatesh	797e42dc24	Watch the ProxyTracker from xDS controller (#18611)	1 year ago
John Murret	0e606504bc	NET-4944 - wire up controllers with proxy tracker (#18603) ... Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>	1 year ago
John Murret	051f250edb	NET-5338 - NET-5338 - Run a v2 mode xds server (#18579) ... * NET-5338 - NET-5338 - Run a v2 mode xds server * fix linting	1 year ago
Semir Patel	067a0112e2	resource: Make resource listbyowner tenancy aware (#18566)	1 year ago
R.B. Boyer	8a931241f2	chore: fix missing/incorrect license headers (#18555)	1 year ago
R.B. Boyer	5b88aae3b4	catalog: validating Protocol and Health enums on Service, Workload, and ServiceEndpoints (#18554)	1 year ago
Ashwin Venkatesh	4f9955d91e	Update trust bundle into proxy-state-template (#18550)	1 year ago
R.B. Boyer	17667a1c75	mesh: adding type aliases for mesh resource usage (#18448) ... Introduces some simple type aliases for DecodedResource[*X] wrappers for each type which cut down on the verbosity	1 year ago
Ashwin Venkatesh	0d60380214	xds controller: resolve ServiceEndpoints references in ProxyStateTemp… (#18544) ... xds controller: resolve ServiceEndpoints references in ProxyStateTemplate	1 year ago
R.B. Boyer	55723c541e	mesh: add validation for the new pbmesh resources (#18410) ... Adds validation for HTTPRoute, GRPCRoute, TCPRoute, DestinationPolicy, and ComputedRoutes.	1 year ago
R.B. Boyer	570c84d032	catalog: add failover mode enum validation (#18545)	1 year ago
Semir Patel	53e28a4963	OSS -> CE (community edition) changes (#18517)	1 year ago
Matt Keeler	547f4f8395	Reduce required type arguments for DecodedResource (#18540)	1 year ago
Iryna Shustava	0b580ffd22	bimapper: fix data race (#18519)	1 year ago
Iryna Shustava	cc596ce772	bimapper: allow to untrack links and support reference or id (#18451)	1 year ago
Ashwin Venkatesh	97b41d946f	Support custom watches on controller (#18439) ... * Support custom watches on controller * refactor mapper methods	1 year ago
Semir Patel	217107f627	resource: Make resource list tenancy aware (#18475)	1 year ago
Poonam Jadhav	f88d4fe28f	Net-2707/list resource endpoint (#18444) ... feat: list resources endpoint	1 year ago
wangxinyi7	cda884ac81	read endpoint (#18268) ... implement http read endpoint to expose resource grpc service read method	1 year ago
Poonam Jadhav	559c61e6b6	Net-2712/resource hcl parsing (#18250) ... * Initial protohcl implementation Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: Daniel Upton <daniel@floppy.co> * resourcehcl: implement resource decoding on top of protohcl Co-authored-by: Daniel Upton <daniel@floppy.co> * fix: resolve ci failures * test: add additional unmarshalling tests * refactor: update function test to clean protohcl package imports --------- Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: Daniel Upton <daniel@floppy.co>	1 year ago
Poonam Jadhav	5717cbd466	Net-2708/delete resource endpoint (#18420) ... * feat: add http delete endpoint for resource service * refactor: clean up	1 year ago
hashicorp-copywrite[bot]	5fb9df1640	[COMPLIANCE] License changes (#18443) ... * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	1 year ago
Semir Patel	bee12c6b1f	resource: Make resource write tenancy aware (#18423)	1 year ago
R.B. Boyer	42efc11b4e	catalog: adding a controller to reconcile FailoverPolicy resources (#18399) ... Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller.	1 year ago
R.B. Boyer	bfc519f293	catalog: add FailoverPolicy mutation and validation hooks (#18390) ... Add most of the validation and mutation hooks for the FailoverPolicy resource.	1 year ago
Matt Keeler	91d331bbaa	Add ServiceEndpoints Mutation hook tests (#18404) ... * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New	1 year ago
Semir Patel	63cc037110	resource: Make resource read tenancy aware (#18397)	1 year ago
R.B. Boyer	1ebd001a07	bimapper: fix a bug and add some more test coverage (#18387)	1 year ago
wangxinyi7	1f28ac2664	expose grpc as http endpoint (#18221) ... expose resource grpc endpoints as http endpoints	1 year ago
R.B. Boyer	9c227e2c36	mesh: adding the protobuf types and resources backing mesh config v2 (#18351)	1 year ago
Iryna Shustava	a33001f4d4	Register ProxyStateTemplate Resource (#18316) ... Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server.	1 year ago
R.B. Boyer	13ce787a3f	resource: adding various helpers for working with resources (#18342) ... This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them.	1 year ago
Semir Patel	efb45fe851	resource: Add scope to resource type registration [NET-4976] (#18214) ... Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped.	1 year ago
Semir Patel	ada767fc9f	resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192)	1 year ago
Semir Patel	003370ded0	Call resource mutate hook before validate hook (NET-4907) (#18178)	1 year ago
Iryna Shustava	c328ba85bd	Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991) ... Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them	1 year ago
Dan Upton	b117eb0126	resource: enforce consistent naming of resource types (#17611) ... For consistency, resource type names must follow these rules: - `Group` must be snake case, and in most cases a single word. - `GroupVersion` must be lowercase, start with a "v" and end with a number. - `Kind` must be pascal case. These were chosen because they map to our protobuf type naming conventions.	1 year ago
Dan Upton	48445dfa55	resource: add `AuthorizerContext` helper method (#17393)	1 year ago
Matt Keeler	37636eab71	Catalog V2 Container Based Integration Test (#17674) ... * Implement the Catalog V2 controller integration container tests This now allows the container tests to import things from the root module. However for now we want to be very restrictive about which packages we allow importing. * Add an upgrade test for the new catalog Currently this should be dormant and not executed. However its put in place to detect breaking changes in the future and show an example of how to do an upgrade test with integration tests structured like catalog v2. * Make testutil.Retry capable of performing cleanup operations These cleanup operations are executed after each retry attempt. * Move TestContext to taking an interface instead of a concrete testing.T This allows this to be used on a retry.R or generally anything that meets the interface. * Move to using TestContext instead of background contexts Also this forces all test methods to implement the Cleanup method now instead of that being an optional interface. Co-authored-by: Daniel Upton <daniel@floppy.co>	1 year ago
Matt Keeler	653a886689	Implement a Catalog Controllers Lifecycle Integration Test (#17435) ... * Implement a Catalog Controllers Lifecycle Integration Test * Prevent triggering the race detector. This allows defining some variables for protobuf constants and using those in comparisons. Without that, something internal in the fmt package ended up looking at the protobuf message size cache and triggering the race detector.	1 year ago
Matt Keeler	a5ba889034	Implement the service endpoints controller (#17216) ... * Add a ReplaceType dep mapper and move them into their own file * Implement the service endpoints controller * Implement a Catalog Controllers Integration Test	2 years ago
Matt Keeler	77f44fa878	Various bits of cleanup detected when using Go Workspaces (#17462) ... TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors. This commit: 1. Fixes envoy-library-references.sh to work again 2. Ensures we are pulling in go-control-plane@v0.11.0 everywhere (previously it was at that version in some modules and others were much older) 3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it. 4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field. 4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules 5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment. 6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible 7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner. 8. Updated linter rules to prevent usage of golang/protobuf 9. Updated a leader peering test to account for an extra colon in a grpc error message.	2 years ago
Dan Stough	d935c7b466	[OSS] gRPC Blocking Queries (#17426) ... * feat: initial grpc blocking queries * changelog and docs update	2 years ago
Matt Keeler	1d6a0c8f21	Add the workload health controller (#17215)	2 years ago
R.B. Boyer	21c6e0e8e6	fix two typos (#17389)	2 years ago
Matt Keeler	d37572bd44	Add a Node health controller (#17214) ... This will aggregate all HealthStatus objects owned by the Node and update the status of the Node with an overall health.	2 years ago
Dan Upton	879b775459	docs: initial documentation for the new State Store (#17315)	2 years ago
Matt Keeler	456156ebec	Add type validations for the catalog resources (#17211) ... Also adding some common resource validation error types to the internal/resource package.	2 years ago
Dan Upton	6c24a66f73	resource: optionally compare timestamps in `EqualStatus` (#17275)	2 years ago
Semir Patel	40eefaba18	Reaper controller for cascading deletes of owner resources (#17256)	2 years ago
Dan Upton	d53a1d4a27	resource: add helpers for more efficiently comparing IDs etc (#17224)	2 years ago
Dan Upton	972998203e	controller: deduplicate items in queue (#17168)	2 years ago
Dan Upton	6e1bc57469	Controller Runtime	2 years ago
Matt Keeler	34915670f2	Register new catalog & mesh protobuf types with the resource registry (#17225)	2 years ago
Semir Patel	991a002fcc	resource: List resources by owner (#17190)	2 years ago
Semir Patel	9fef1c7f17	Create tombstone on resource `Delete` (#17108)	2 years ago
Dan Upton	eeaa636164	Cleanup from unblocking the pipeline 🧹 (#17121)	2 years ago
Semir Patel	e7bb8fdf15	Fix or disable pipeline breaking changes that made it into main in last day or so (#17130) ... * Fix straggler from renaming Register->RegisterTypes * somehow a lint failure got through previously * Fix lint-consul-retry errors * adding in fix for success jobs getting skipped. (#17132) * Temporarily disable inmem backend conformance test to get green pipeline * Another test needs disabling --------- Co-authored-by: John Murret <john.murret@hashicorp.com>	2 years ago
Dan Upton	b9c485dcb8	Controller Supervision (#17016)	2 years ago
Dan Upton	ba4a314772	storage: fix bug where WatchList would (rarely) return duplicate events (#17067)	2 years ago
hashicorp-copywrite[bot]	9f81fc01e9	[COMPLIANCE] Add Copyright and License Headers (#16854) ... Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>	2 years ago
Semir Patel	2f7d591702	Tenancy wildcard validaton for `Write`, `Read`, and `Delete` endpoints (#17004)	2 years ago
Semir Patel	b8c9e133be	Add mutate hook to `Write` endpoint (#16958)	2 years ago
Semir Patel	3b83c7ee9a	Enforce ACLs on resource `Write` and `Delete` endpoints (#16956)	2 years ago
Dan Upton	d595e6ade9	resource: `WriteStatus` endpoint (#16886)	2 years ago
Semir Patel	317240fca7	Resource validation hook for `Write` endpoint (#16950)	2 years ago
Semir Patel	686f49346c	Check acls on resource `Read`, `List`, and `WatchList` (#16842)	2 years ago
Dan Upton	4fa2537b3b	Resource `Write` endpoint (#16786)	2 years ago
Dan Upton	671d5825ca	Raft storage backend (#16619)	2 years ago
Dan Upton	651549c97d	storage: fix resource leak in Watch (#16817)	2 years ago
Ronald	b64674623e	Copyright headers for missing files/folders (#16708) ... * copyright headers for agent folder	2 years ago
Dan Upton	81df781e5f	Add storage backend interface and in-memory implementation (#16538) ... Introduces `storage.Backend`, which will serve as the interface between the Resource Service and the underlying storage system (Raft today, but in the future, who knows!). The primary design goal of this interface is to keep its surface area small, and push as much functionality as possible into the layers above, so that new implementations can be added with little effort, and easily proven to be correct. To that end, we also provide a suite of "conformance" tests that can be run against a backend implementation to check it behaves correctly. In this commit, we introduce an initial in-memory storage backend, which is suitable for tests and when running Consul in development mode. This backend is a thin wrapper around the `Store` type, which implements a resource database using go-memdb and our internal pub/sub system. `Store` will also be used to handle reads in our Raft backend, and in the future, used as a local cache for external storage systems.	2 years ago
Poonam Jadhav	9c64731a56	feat: add category annotation to RPC and gRPC methods (#16646)	2 years ago
Semir Patel	5a3fec6238	Basic resource type registry (#16622)	2 years ago
Matt Keeler	84156afe87	Remove private prefix from proto-gen-rpc-glue e2e test (#16433)	2 years ago
Matt Keeler	085c0addc0	Protobuf Refactoring for Multi-Module Cleanliness (#16302) ... Protobuf Refactoring for Multi-Module Cleanliness This commit includes the following: Moves all packages that were within proto/ to proto/private Rewrites imports to account for the packages being moved Adds in buf.work.yaml to enable buf workspaces Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes) Why: In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage. There were some recent changes to have our own ratelimiting annotations. The two combined were not working when I was trying to use them together (attempting to rebase another branch) Buf workspaces should be the solution to the problem Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root. This resulted in proto file name conflicts in the Go global protobuf type registry. The solution to that was to add in a private/ directory into the path within the proto/ directory. That then required rewriting all the imports. Is this safe? AFAICT yes The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc) Other than imports, there were no changes to any generated code as a result of this.	2 years ago
Matt Keeler	5afd4657ec	Protobuf Modernization (#15949) ... * Protobuf Modernization Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf Marshallers (protobuf and json) needed some changes to account for different APIs. Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package. This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces. * Fix go-mod-tidy make target to work on all modules	2 years ago
Dan Upton	ee2d47da83	Fix missing comment in generated enterprise rate-limit file (#15895) ... Fixes bug in #15564 where gofmt would strip out the generated code warning comment because it was on the same line as the build tag.	2 years ago
Dan Upton	d4c435856b	grpc: `protoc` plugin for generating gRPC rate limit specifications (#15564) ... Adds automation for generating the map of `gRPC Method Name → Rate Limit Type` used by the middleware introduced in #15550, and will ensure we don't forget to add new endpoints. Engineers must annotate their RPCs in the proto file like so: ``` rpc Foo(FooRequest) returns (FooResponse) { option (consul.internal.ratelimit.spec) = { operation_type: READ, }; } ``` When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will be installed that writes rate-limit specs as a JSON array to a file called `.ratelimit.tmp` (one per protobuf package/directory). After running Buf, `make proto` will execute a post-process script that will ingest all of the `.ratelimit.tmp` files and generate a Go file containing the mappings in the `agent/grpc-middleware` package. In the enterprise repository, it will write an additional file with the enterprise-only endpoints. If an engineer forgets to add the annotation to a new RPC, the plugin will return an error like so: ``` RPC Foo is missing rate-limit specification, fix it with: import "proto-public/annotations/ratelimit/ratelimit.proto"; service Bar { rpc Foo(...) returns (...) { option (hashicorp.consul.internal.ratelimit.spec) = { operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT, }; } } ``` In the future, this annotation can be extended to support rate-limit category (e.g. KV vs Catalog) and to determine the retry policy.	2 years ago
Chris S. Kim	cc819ad83b	[OSS] Add boilerplate for proto files implementing BlockableQuery (#15554)	2 years ago
Kyle Schochenmaier	bf0f61a878	removes ioutil usage everywhere which was deprecated in go1.16 (#15297) ... * update go version to 1.18 for api and sdk, go mod tidy * removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward. Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	2 years ago
Chris S. Kim	bde57c0dd0	Regenerate files according to 1.19.2 formatter	2 years ago
Chris S. Kim	3d05a8e9af	Remove unused methods from template	2 years ago
Chris S. Kim	29a297d3e9	Refactor client RPC timeouts (#14965) ... Fix an issue where rpc_hold_timeout was being used as the timeout for non-blocking queries. Users should be able to tune read timeouts without fiddling with rpc_hold_timeout. A new configuration `rpc_read_timeout` is created. Refactor some implementation from the original PR 11500 to remove the misleading linkage between RPCInfo's timeout (used to retry in case of certain modes of failures) and the client RPC timeouts.	2 years ago
Paul Glass	77afe0e76e	Extract AWS auth implementation out of Consul (#13760)	2 years ago
alex	7c0daeade8	fix leader annotation (#13786) ... Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>	2 years ago
alex	b7043f7150	peering: add warning about AllowStaleRead (#13768)	2 years ago
Evan Culver	88449b1f1b	internal: port RPC glue changes from Enterprise (#13034) ... Co-authored-by: R.B. Boyer <rb@hashicorp.com>	3 years ago
Will Jordan	c48120d005	Add timeout to Client RPC calls (#11500) ... Adds a timeout (deadline) to client RPC calls, so that streams will no longer hang indefinitely in unstable network conditions. Co-authored-by: kisunji <ckim@hashicorp.com>	3 years ago
Paul Glass	d79ca2f024	acl: Fix tag parsing for IAM users and roles in IAM auth method (#12797) ... * acl: Fix tag parsing on IAM users and roles in IAM auth method * Add changelog	3 years ago
Paul Glass	99f373dde4	acl: Adjust region handling in AWS IAM auth method (#12774) ... * acl: Adjust region handling in AWS IAM auth method	3 years ago
Paul Glass	706c844423	Add IAM Auth Method (#12583) ... This adds an aws-iam auth method type which supports authenticating to Consul using AWS IAM identities. Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	3 years ago
R.B. Boyer	4676960b93	proto-gen-rpc-glue: support QueryMeta and QueryOptions (#12637)	3 years ago
R.B. Boyer	dc023cb0dd	proto-gen-rpc-glue: use a shallow copy of proto/pbcommon instead of a consul dependency (#12634)	3 years ago
R.B. Boyer	9736e33897	proto-gen-rpc-glue: fix behavior of renamed fields (#12633)	3 years ago
R.B. Boyer	3d725a1762	regenerate rpc glue stubs in protobuf files using comments (#12625)	3 years ago
Daniel Nephin	e8312d6b5a	testing: remove unnecessary calls to freeport ... Previously we believe it was necessary for all code that required ports to use freeport to prevent conflicts. https://github.com/dnephin/freeport-test shows that it is actually save to use port 0 (`127.0.0.1:0`) as long as it is passed directly to `net.Listen`, and the listener holds the port for as long as it is needed. This works because freeport explicitly avoids the ephemeral port range, and port 0 always uses that range. As you can see from the test output of https://github.com/dnephin/freeport-test, the two systems never use overlapping ports. This commit converts all uses of freeport that were being passed directly to a net.Listen to use port 0 instead. This allows us to remove a bit of wrapping we had around httptest, in a couple places.	3 years ago
Daniel Nephin	56f9238d15	go-sso: remove returnFunc now that freeport handles return	3 years ago
Evan Culver	c4c833fd0e	sso/oidc: add support for acr_values request parameter (#11026) ... Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	3 years ago
Daniel Nephin	a6000e6ad8	state: add a regression test for state store schema ... To allow the index to be refactored without accidental changes. To update the expected value run: 'go test ./agent/consul/state -update'	4 years ago
Daniel Nephin	b9e60c0775	testing: skip slow tests with -short ... Add a skip condition to all tests slower than 100ms. This change was made using `gotestsum tool slowest` with data from the last 3 CI runs of master. See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests With this change: ``` $ time go test -count=1 -short ./agent ok github.com/hashicorp/consul/agent 0.743s real 0m4.791s $ time go test -count=1 -short ./agent/consul ok github.com/hashicorp/consul/agent/consul 4.229s real 0m8.769s ```	4 years ago
Matt Keeler	51c3a605ad	Merge pull request #8035 from hashicorp/feature/auto-config/server-rpc	5 years ago
Daniel Nephin	5afcf5c1bc	Merge pull request #8034 from hashicorp/dnephin/add-linter-staticcheck-4 ... ci: enable SA4006 staticcheck check and add ineffassign	5 years ago
Matt Keeler	9b01f9423c	Implement the insecure version of the Cluster.AutoConfig RPC endpoint ... Right now this is only hooked into the insecure RPC server and requires JWT authorization. If no JWT authorizer is setup in the configuration then we inject a disabled “authorizer” to always report that JWT authorization is disabled.	5 years ago
Daniel Nephin	068b43df90	Enable gofmt simplify ... Code changes done automatically with 'gofmt -s -w'	5 years ago
Daniel Nephin	cb050b280c	ci: enable SA4006 staticcheck check ... And fix the 'value not used' issues. Many of these are not bugs, but a few are tests not checking errors, and one appears to be a missed error in non-test code.	5 years ago
Daniel Nephin	600645b5f9	Add unconvert linter ... To find unnecessary type convertions	5 years ago
R.B. Boyer	1efafd7523	acl: add auth method for JWTs (#7846)	5 years ago
Jeff Mitchell	4243c3ae42	Move internal/ to sdk/ (#5568) ... * Move internal/ to sdk/ * Add a readme to the SDK folder	6 years ago
Jeff Mitchell	bb9af0dc37	Bump cleanhttp to be the same version as the other modules (#5563)	6 years ago
Jeff Mitchell	47c390025b	Convert to Go Modules (#5517) ... * First conversion * Use serf 0.8.2 tag and associated updated deps * * Move freeport and testutil into internal/ * Make internal/ its own module * Update imports * Add replace statements so API and normal Consul code are self-referencing for ease of development * Adapt to newer goe/values * Bump to new cleanhttp * Fix ban nonprintable chars test * Update lock bad args test The error message when the duration cannot be parsed changed in Go 1.12 (ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test. * Update another test as well * Bump travis * Bump circleci * Bump go-discover and godo to get rid of launchpad dep * Bump dockerfile go version * fix tar command * Bump go-cleanhttp	6 years ago