Commit Graph

42 Commits (4e9c624728a9f923169e7b6fc2cf0fc690493ff1)

Author SHA1 Message Date
hc-github-team-consul-core 82e8aa2e92
Backport of Deflake the catalog v2beta1 integration tests into release/1.17.x (#20280)
Deflake the catalog v2beta1 integration tests (#20278)

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2024-01-19 19:49:56 +00:00
hc-github-team-consul-core b0d7faa9bd
Backport of test: add test helper to detect if the current build is enterprise into release/1.17.x (#19612)
test: add test helper to detect if the current build is enterprise (#19201)

This can help conditionally change test behavior if the test is executing in enterprise without the need to split the test with build tags.

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2023-11-13 17:12:25 +00:00
hc-github-team-consul-core da2a57239e
Backport of Introduce randomized timings and reproducible randomization into controller integration tests. into release/1.17.x (#19575)
Introduce randomized timings and reproducible randomization into controller integration tests.

As the V2 architecture hinges on eventual consistency and controllers reconciling the existing state in response to writes, there are potential issues we could run into regarding ordering and timing of operations. We want to be able to guarantee that given a set of resources the system will always eventually get to the desired correct state. The order of resource writes and delays in performing those writes should not alter the final outcome of reaching the desired state.

To that end, this commit introduces arbitrary randomized delays before performing resources writes into the `resourcetest.Client`. Its `PublishResources` method was already randomizing the order of resource writes. By default, no delay is added to normal writes and deletes but tests can opt-in via either passing hard coded options when creating the `resourcetest.Client` or using the `resourcetest.ConfigureTestCLIFlags` function to allow processing of CLI parameters.

In addition to allowing configurability of the request delay min and max, the client also has a configurable random number generator seed. When Using the CLI parameter helpers, a test log will be written noting the currently used settings. If the test fails then you can reproduce the same delays and order randomizations by providing the seed during the previous test failure.

# Conflicts:
#	internal/catalog/catalogtest/run_test.go
#	internal/catalog/catalogtest/test_integration_v2beta1.go
#	test/integration/consul-container/test/catalog/catalog_test.go

Co-authored-by: Matt Keeler <mjkeeler7@gmail.com>
2023-11-08 16:39:28 -05:00
hc-github-team-consul-core e6c6814c13
Backport of [NET-6305] xds: Ensure v2 route match and protocol are populated for gRPC into release/1.17.x (#19366)
* backport of commit 0a8fe0d94f

* backport of commit 74bf2aa8b0

---------

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2023-10-25 19:12:18 +00:00
hc-github-team-consul-core 8a3a15eb8f
Backport of acls,catalog,mesh: properly authorize workload selectors on writes into release/1.17.x (#19296)
backport of commit 6350a814db

Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
2023-10-19 17:28:03 +00:00
hc-github-team-consul-core 168a640a90
Backport of catalog: add FailoverPolicy ACL hook tenancy test into release/1.17.x (#19223)
backport of commit b98d6458e3

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2023-10-16 19:47:39 +00:00
hc-github-team-consul-core 689f32c59d
Backport of catalog, mesh: implement missing ACL hooks into release/1.17.x (#19212)
catalog, mesh: implement missing ACL hooks (#19143)

This change adds ACL hooks to the remaining catalog and mesh resources, excluding any computed ones. Those will for now continue using the default operator:x permissions.

It refactors a lot of the common testing functions so that they can be re-used between resources.

There are also some types that we don't yet support (e.g. virtual IPs) that this change adds ACL hooks to for future-proofing.

Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com>
2023-10-14 01:50:22 +00:00
hc-github-team-consul-core 41a986c6e0
Backport of mesh: add more validations to Destinations resource into release/1.17.x (#19211)
backport of commit f6c7c4ddc1

Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
2023-10-13 23:08:06 +00:00
R.B. Boyer 99f7a1219e
catalog: add metadata filtering to refine workload selectors (#19198)
This implements the Filter field on pbcatalog.WorkloadSelector to be
a post-fetch in-memory filter using the https://github.com/hashicorp/go-bexpr
expression language to filter resources based on their envelope metadata fields.

All existing usages of WorkloadSelector should be able to make use of the filter.
2023-10-13 13:37:42 -05:00
Iryna Shustava 54a12ab3c9
mesh: sidecar proxy controller improvements (#19083)
This change builds on #19043 and #19067 and updates the sidecar controller to use those computed resources. This achieves several benefits:

   * The cache is now simplified which helps us solve for previous bugs (such as multiple Upstreams/Destinations targeting the same service would overwrite each other)
   * We no longer need proxy config cache
   * We no longer need to do merging of proxy configs as part of the controller logic
   * Controller watches are simplified because we no longer need to have complex mapping using cache and can instead use the simple ReplaceType mapper.

It also makes several other improvements/refactors:

  * Unifies all caches into one. This is because originally the caches were more independent, however, now that they need to interact with each other it made sense to unify them where sidecar proxy controller uses one cache with 3 bimappers
   * Unifies cache and mappers. Mapper already needed all caches anyway and so it made sense to make the cache do the mapping also now that the cache is unified.
   * Gets rid of service endpoints watches. This was needed to get updates in a case when service's identities have changed and we need to update proxy state template's spiffe IDs for those destinations. This will however generate a lot of reconcile requests for this controller as service endpoints objects can change a lot because they contain workload's health status. This is solved by adding a status to the service object tracking "bound identities" and have service endpoints controller update it. Having service's status updated allows us to get updates in the sidecar proxy controller because it's already watching service objects
   * Add a watch for workloads. We need it so that we get updates if workload's ports change. This also ensures that we update cached identities in case workload's identity changes.
2023-10-12 13:20:13 -06:00
R.B. Boyer 29ba5b5c79
catalog: block unsupported failover policy settings for now (#19168) 2023-10-12 11:13:56 -05:00
John Murret 6da4798e05
NET-5799 - ensure catalog controllers and dependency mappers function correctly for tenancy fields (#19142)
* use bimapper

* WIP

* clean up

* PR feedback
2023-10-12 02:07:50 +00:00
Iryna Shustava c35df12c95
mesh: Add ComputedProxyConfiguration and a controller that computes it. (#19043)
* Introduce a new type `ComputedProxyConfiguration` and add a controller for it. This is needed for two reasons. The first one is that external integrations like kubernetes may need to read the fully computed and sorted proxy configuration per workload. The second reasons is that it makes sidecar-proxy controller logic quite a bit simpler as it no longer needs to do this.
* Generalize workload selection mapper and fix a bug where it would delete IDs from the tree if only one is left after a removal is done.
2023-10-10 17:34:53 -06:00
Iryna Shustava e6b724d062
catalog,mesh,auth: Move resource types to the proto-public module (#18935) 2023-09-22 15:50:56 -06:00
Iryna Shustava d88888ee8b
catalog,mesh,auth: Bump versions to v2beta1 (#18930) 2023-09-22 10:51:15 -06:00
R.B. Boyer de231bbbdd
catalog: fix for new method argument (#18978) 2023-09-22 10:42:16 -05:00
R.B. Boyer ec6189fd2f
catalog: add ACL checks for FailoverPolicy resources (#18919)
FailoverPolicy resources are name-aligned with the Service they control.
They also contain a list of possible failover destinations that are References
 to other Services.

The ACLs should be:

- list: (default)
- read: service:<resource_name>:read
- write: service:<resource_name>:write + service:<destination_name>:read (for any destination)
2023-09-22 09:59:14 -05:00
R.B. Boyer d574473fd1
mesh: make FailoverPolicy work in xdsv2 and ProxyStateTemplate (#18900)
Ensure that configuring a FailoverPolicy for a service that is reachable via a xRoute or a direct upstream causes an envoy aggregate cluster to be created for the original cluster name, but with separate clusters for each one of the possible destinations.
2023-09-20 11:59:01 -05:00
R.B. Boyer b4d5178e5c
catalog: normalize/default/validate tenancy components of FailoverPolicy internal References (#18825)
FailoverPolicy resources contain inner Reference fields. We want to ensure 
that components of those reference Tenancy fields left unspecified are defaulted 
using the tenancy of the enclosing FailoverPolicy resource.

As the underlying helper being used to do the normalization calls the function 
modified in #18822, it also means that the PeerName field will be set to "local" for 
now automatically to avoid "local" != "" issues downstream.
2023-09-18 14:59:08 -05:00
Iryna Shustava a89938e0c1
catalog: Default protocol to tcp in catalog.Service if unspecified (#18832) 2023-09-15 15:11:56 -06:00
Iryna Shustava 7f2a1d9812
catalog: service endpoints inherits protocol from service when workload doesn't have one (#18792) 2023-09-14 08:52:12 -06:00
John Murret 3e78b4cf34
Prefix sidecar proxy test files with source and destination. (#18620)
* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Address PR comments

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* PR review comments

* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Make sure endpoint refs route to mesh port instead of an app port

* Address PR comments

* fixing copyright

* tidy imports

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* tidy imports

* add copyright headers

* Prefix sidecar proxy test files with source and destination.

* Update controller_test.go

---------

Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
2023-09-08 17:28:55 -06:00
Iryna Shustava 3c70e14713
sidecar-proxy controller: L4 controller with explicit upstreams (NET-3988) (#18352)
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* This commit also changes service endpoints to include workload identity. This made the implementation a bit easier as we don't need to look up as many workloads and instead rely on endpoints data.
2023-09-07 09:37:15 -06:00
Semir Patel b96cff7436
resource: Require scope for resource registration (#18635) 2023-09-01 09:44:53 -05:00
Michael Zalimeni 699aa47416
fix: make UNSPECIFIED protocol pass validation (#18634)
We explicitly enumerate the allowed protocols in validation, so this
change is necessary to use the new enum value.

Also add tests for enum validators to ensure they stay aligned to
protos unless we explicitly want them to diverge.
2023-08-31 19:59:29 +00:00
Dhia Ayachi f8d77f027a
delete all v2 resources type when deleting a namespace (CE) (#18621)
* add namespace scope to ServiceV1Alpha1Type

* add CE portion of namespace deletion
2023-08-31 10:18:25 -04:00
R.B. Boyer 5b88aae3b4
catalog: validating Protocol and Health enums on Service, Workload, and ServiceEndpoints (#18554) 2023-08-22 15:16:55 -05:00
R.B. Boyer 570c84d032
catalog: add failover mode enum validation (#18545) 2023-08-22 11:09:37 -05:00
Matt Keeler 547f4f8395
Reduce required type arguments for DecodedResource (#18540) 2023-08-21 20:20:19 -04:00
Iryna Shustava cc596ce772
bimapper: allow to untrack links and support reference or id (#18451) 2023-08-17 18:03:05 -06:00
hashicorp-copywrite[bot] 5fb9df1640
[COMPLIANCE] License changes (#18443)
* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* add missing license headers

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
2023-08-11 09:12:13 -04:00
R.B. Boyer 42efc11b4e
catalog: adding a controller to reconcile FailoverPolicy resources (#18399)
Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller.
2023-08-09 11:02:17 -05:00
R.B. Boyer bfc519f293
catalog: add FailoverPolicy mutation and validation hooks (#18390)
Add most of the validation and mutation hooks for the FailoverPolicy resource.
2023-08-08 14:42:43 -05:00
Matt Keeler 91d331bbaa
Add ServiceEndpoints Mutation hook tests (#18404)
* Add ServiceEndpoints Mutation hook tests

* Move endpoint owner validation into the validation hook

Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New
2023-08-08 15:22:14 -04:00
R.B. Boyer 9c227e2c36
mesh: adding the protobuf types and resources backing mesh config v2 (#18351) 2023-08-03 13:42:04 -05:00
Matt Keeler 653a886689
Implement a Catalog Controllers Lifecycle Integration Test (#17435)
* Implement a Catalog Controllers Lifecycle Integration Test

* Prevent triggering the race detector.

This allows defining some variables for protobuf constants and using those in comparisons. Without that, something internal in the fmt package ended up looking at the protobuf message size cache and triggering the race detector.
2023-06-16 12:58:53 -04:00
Matt Keeler a5ba889034
Implement the service endpoints controller (#17216)
* Add a ReplaceType dep mapper and move them into their own file

* Implement the service endpoints controller

* Implement a Catalog Controllers Integration Test
2023-06-06 17:09:48 -04:00
Matt Keeler 1d6a0c8f21
Add the workload health controller (#17215) 2023-05-19 13:53:29 -04:00
R.B. Boyer 21c6e0e8e6
fix two typos (#17389) 2023-05-17 08:50:26 -07:00
Matt Keeler d37572bd44
Add a Node health controller (#17214)
This will aggregate all HealthStatus objects owned by the Node and update the status of the Node with an overall health.
2023-05-15 09:55:03 -04:00
Matt Keeler 456156ebec
Add type validations for the catalog resources (#17211)
Also adding some common resource validation error types to the internal/resource package.
2023-05-12 09:24:55 -04:00
Matt Keeler 34915670f2
Register new catalog & mesh protobuf types with the resource registry (#17225) 2023-05-08 15:36:35 -04:00