212 Commits (CC-5545/upgrade-hds-packages)

Author	SHA1	Message	Date
Semir Patel	830c4ea81c	v2tenancy: cluster scoped reads (#19082)	1 year ago
Chris S. Kim	92ce814693	Remove old build tags (#19128)	1 year ago
skpratt	1fda2965e8	Allow empty data writes for resources (#18819) ... * allow nil data writes for resources * update demo to test valid type with no data	1 year ago
Nitya Dhanushkodi	78b170ad50	xds controller: setup watches for and compute leaf cert references in ProxyStateTemplate, and wire up leaf cert manager dependency (#18756) ... * Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager. The leaf cert logic in the controller: * Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too). * Gets the leaf cert from the leaf cert cache * Stores the leaf cert in the ProxyState that's pushed to xds * For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource. The wrapper allows mapping events to resources (as opposed to mapping resources to resources) The controller tests: Unit: Ensure that we resolve leaf cert references Lifecycle: Ensure that when the CA is updated, the leaf cert is as well Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id. * Pulls out some leaf cert test helpers into a helpers file so it can be used in the xds controller tests. * Wires up leaf cert manager dependency * Support getting token from proxytracker * Add workload identity spiffe id type to the authorize and sign functions --------- Co-authored-by: John Murret <john.murret@hashicorp.com>	1 year ago
R.B. Boyer	a69e901660	xds: update golden tests to be deterministic (#18707)	1 year ago
Nathan Coleman	e5d26a13cd	NET-5530 Support response header modifiers on http-route config entry (#18646) ... * Add response header filters to http-route config entry definitions * Map response header filters from config entry when constructing route destination * Support response header modifiers at the service level as well * Update protobuf definitions * Update existing unit tests * Add response filters to route consolidation logic * Make existing unit tests more robust * Add missing docstring * Add changelog entry * Add response filter modifiers to existing integration test * Add more robust testing for response header modifiers in the discovery chain * Add more robust testing for request header modifiers in the discovery chain * Modify test to verify that service filter modifiers take precedence over rule filter modifiers	1 year ago
Phil Porada	7ea986783d	Add TCP+TLS Healthchecks (#18381) ... * Begin adding TCPUseTLS * More TCP with TLS plumbing * Making forward progress * Keep on adding TCP+TLS support for healthchecks * Removed too many lines * Unit tests for TCP+TLS * Update tlsutil/config.go Co-authored-by: Samantha <hello@entropy.cat> * Working on the tcp+tls unit test * Updated the runtime integration tests * Progress * Revert this file back to HEAD * Remove debugging lines * Implement TLS enabled TCP socket server and make a successful TCP+TLS healthcheck on it * Update docs * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update agent/checks/check.go Co-authored-by: Samantha <hello@entropy.cat> * Address comments * Remove extraneous bracket * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update the mockTLSServer * Remove trailing newline * Address comments * Fix merge problem * Add changelog entry --------- Co-authored-by: Samantha <hello@entropy.cat>	1 year ago
Semir Patel	b96cff7436	resource: Require scope for resource registration (#18635)	1 year ago
Chris S. Kim	ecdcde4309	CE commit (#18583)	1 year ago
Chris S. Kim	82993fcc4f	CE port of enterprise extension (#18572) ... CE commit	1 year ago
Semir Patel	53e28a4963	OSS -> CE (community edition) changes (#18517)	1 year ago
R.B. Boyer	66bcaa3c8f	build: upgrade to latest buf v1.26.0 (#18426)	1 year ago
hashicorp-copywrite[bot]	5fb9df1640	[COMPLIANCE] License changes (#18443) ... * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	1 year ago
John Maguire	df11e4e7b4	APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) ... * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file	1 year ago
John Maguire	6c8ca0f89d	NET-4984: Update APIGW Config Entries for JWT Auth (#18366) ... * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types	1 year ago
R.B. Boyer	42efc11b4e	catalog: adding a controller to reconcile FailoverPolicy resources (#18399) ... Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller.	1 year ago
sarahalsmiller	e235c8be3c	NET-5115 Add retry + timeout filters for api-gateway (#18324) ... * squash, implement retry/timeout in consul core * update tests	1 year ago
Semir Patel	63cc037110	resource: Make resource read tenancy aware (#18397)	1 year ago
cui fliter	18a5edd232	docs: Fix some comments (#17118) ... Signed-off-by: cui fliter <imcusg@gmail.com>	1 year ago
Ronald	80394278b8	Expose JWKS cluster config through JWTProviderConfigEntry (#17978) ... * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA	1 year ago
Matt Keeler	653a886689	Implement a Catalog Controllers Lifecycle Integration Test (#17435) ... * Implement a Catalog Controllers Lifecycle Integration Test * Prevent triggering the race detector. This allows defining some variables for protobuf constants and using those in comparisons. Without that, something internal in the fmt package ended up looking at the protobuf message size cache and triggering the race detector.	1 year ago
Eric Haberkorn	779647b948	Add Envoy and Consul version constraints to Envoy extensions (#17612)	1 year ago
Matt Keeler	77f44fa878	Various bits of cleanup detected when using Go Workspaces (#17462) ... TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors. This commit: 1. Fixes envoy-library-references.sh to work again 2. Ensures we are pulling in go-control-plane@v0.11.0 everywhere (previously it was at that version in some modules and others were much older) 3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it. 4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field. 4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules 5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment. 6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible 7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner. 8. Updated linter rules to prevent usage of golang/protobuf 9. Updated a leader peering test to account for an extra colon in a grpc error message.	2 years ago
Dan Stough	d935c7b466	[OSS] gRPC Blocking Queries (#17426) ... * feat: initial grpc blocking queries * changelog and docs update	2 years ago
R.B. Boyer	e00280e7df	prototest: fix early return condition in AssertElementsMatch (#17416)	2 years ago
Ronald	113202d541	JWT Authentication with service intentions: xds package update (#17414) ... * JWT Authentication with service intentions: update xds package to translate config to envoy	2 years ago
Semir Patel	5eaeb7b8e5	Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks (#15979) ... * Add MaxEjectionPercent to config entry * Add BaseEjectionTime to config entry * Add MaxEjectionPercent and BaseEjectionTime to protobufs * Add MaxEjectionPercent and BaseEjectionTime to api * Fix integration test breakage * Verify MaxEjectionPercent and BaseEjectionTime in integration test upstream confings * Website docs for MaxEjectionPercent and BaseEjection time * Add `make docs` to browse docs at http://localhost:3000 * Changelog entry * so that is the difference between consul-docker and dev-docker * blah * update proto funcs * update proto --------- Co-authored-by: Maliz <maliheh.monshizadeh@hashicorp.com>	2 years ago
Eric Haberkorn	b1fae05983	Add sameness groups to service intentions. (#17064)	2 years ago
Ronald	e07c09dbfb	Fix generated proto files (#17063) ... * [COMPLIANCE] Add Copyright and License Headers * generate proto --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	2 years ago
hashicorp-copywrite[bot]	9f81fc01e9	[COMPLIANCE] Add Copyright and License Headers (#16854) ... Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>	2 years ago
Paul Glass	f4406e69b9	[NET-3091] Update service intentions to support jwt provider references (#17037) ... * [NET-3090] Add new JWT provider config entry * Add initial test cases * update validations for jwt-provider config entry fields * more validation * start improving tests * more tests * Normalize * Improve tests and move validate fns * usage test update * Add split between ent and oss for partitions * fix lint issues * Added retry backoff, fixed tests, removed unused defaults * take into account default partitions * use countTrue and add aliases * omit audiences if empty * fix failing tests * add omit-entry * Add JWT intentions * generate proto * fix deep copy issues * remove extra field * added some tests * more tests * add validation for creating existing jwt * fix nil issue * More tests, fix conflicts and improve memdb call * fix namespace * add aliases * consolidate errors, skip duplicate memdb calls * reworked iteration over config entries * logic improvements from review --------- Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>	2 years ago
Paul Glass	77ecff3209	Permissive mTLS (#17035) ... This implements permissive mTLS , which allows toggling services into "permissive" mTLS mode. Permissive mTLS mode allows incoming "non Consul-mTLS" traffic to be forward unmodified to the application. * Update service-defaults and proxy-defaults config entries with a MutualTLSMode field * Update the mesh config entry with an AllowEnablingPermissiveMutualTLS field and implement the necessary validation. AllowEnablingPermissiveMutualTLS must be true to allow changing to MutualTLSMode=permissive, but this does not require that all proxy-defaults and service-defaults are currently in strict mode. * Update xDS listener config to add a "permissive filter chain" when MutualTLSMode=permissive for a particular service. The permissive filter chain matches incoming traffic by the destination port. If the destination port matches the service port from the catalog, then no mTLS is required and the traffic sent is forwarded unmodified to the application.	2 years ago
Derek Menteer	87324c9ec8	Add PrioritizeByLocality to config entries. (#17007) ... This commit adds the PrioritizeByLocality field to both proxy-config and service-resolver config entries for locality-aware routing. The field is currently intended for enterprise only, and will be used to enable prioritization of service-mesh connections to services based on geographical region / zone.	2 years ago
Michael Wilkerson	0dd4ea2033	* added Sameness Group to proto files (#16998) ... - added Sameness Group to config entries - added Sameness Group to subscriptions * generated proto files * added Sameness Group events to the state store - added test cases * Refactored health RPC Client - moved code that is common to rpcclient under rpcclient common.go. This will help set us up to support future RPC clients * Refactored proxycfg glue views - Moved views to rpcclient config entry. This will allow us to reuse this code for a config entry client * added config entry RPC Client - Copied most of the testing code from rpcclient/health * hooked up new rpcclient in agent * fixed documentation and comments for clarity	2 years ago
Dan Upton	4fa2537b3b	Resource `Write` endpoint (#16786)	2 years ago
Dan Upton	671d5825ca	Raft storage backend (#16619)	2 years ago
Eric Haberkorn	0d1d2fc4c9	add order by locality failover to Consul enterprise (#16791)	2 years ago
Ronald	b64674623e	Copyright headers for missing files/folders (#16708) ... * copyright headers for agent folder	2 years ago
Dhia Ayachi	10df4d83aa	add ip rate limiter controller OSS parts (#16790)	2 years ago
Dan Upton	81df781e5f	Add storage backend interface and in-memory implementation (#16538) ... Introduces `storage.Backend`, which will serve as the interface between the Resource Service and the underlying storage system (Raft today, but in the future, who knows!). The primary design goal of this interface is to keep its surface area small, and push as much functionality as possible into the layers above, so that new implementations can be added with little effort, and easily proven to be correct. To that end, we also provide a suite of "conformance" tests that can be run against a backend implementation to check it behaves correctly. In this commit, we introduce an initial in-memory storage backend, which is suitable for tests and when running Consul in development mode. This backend is a thin wrapper around the `Store` type, which implements a resource database using go-memdb and our internal pub/sub system. `Store` will also be used to handle reads in our Raft backend, and in the future, used as a local cache for external storage systems.	2 years ago
Poonam Jadhav	9c64731a56	feat: add category annotation to RPC and gRPC methods (#16646)	2 years ago
Eric Haberkorn	eaa39f4ef5	add sameness group support to service resolver failover and redirects (#16664)	2 years ago
Eric Haberkorn	57e2493415	allow setting locality on services and nodes (#16581)	2 years ago
Eric Haberkorn	dbaf8bf49c	add agent locality and replicate it across peer streams (#16522)	2 years ago
Eric Haberkorn	5f81662066	Add support for failover policies (#16505)	2 years ago
Andrew Stucki	4b661d1e0c	Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable (#16495) ... * Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable * Regenerate golden files * Add RequestTimeout field * Add changelog entry	2 years ago
Andrew Stucki	0972697661	[API Gateway] Various fixes for Config Entry fields (#16347) ... * [API Gateway] Various fixes for Config Entry fields * simplify logic per PR review	2 years ago
Matt Keeler	085c0addc0	Protobuf Refactoring for Multi-Module Cleanliness (#16302) ... Protobuf Refactoring for Multi-Module Cleanliness This commit includes the following: Moves all packages that were within proto/ to proto/private Rewrites imports to account for the packages being moved Adds in buf.work.yaml to enable buf workspaces Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes) Why: In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage. There were some recent changes to have our own ratelimiting annotations. The two combined were not working when I was trying to use them together (attempting to rebase another branch) Buf workspaces should be the solution to the problem Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root. This resulted in proto file name conflicts in the Go global protobuf type registry. The solution to that was to add in a private/ directory into the path within the proto/ directory. That then required rewriting all the imports. Is this safe? AFAICT yes The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc) Other than imports, there were no changes to any generated code as a result of this.	2 years ago
Nathan Coleman	dd0ca4825c	Add inline-certificate as possible payload of config-entry wrapper (#16254) ... Co-authored-by: Andrew Stucki <3577250+andrewstucki@users.noreply.github.com>	2 years ago
Thomas Eckert	e81a0c2855	API Gateway to Ingress Gateway Snapshot Translation and Routes to Virtual Routers and Splitters (#16127) ... * Stub proxycfg handler for API gateway * Add Service Kind constants/handling for API Gateway * Begin stubbing for SDS * Add new Secret type to xDS order of operations * Continue stubbing of SDS * Iterate on proxycfg handler for API gateway * Handle BoundAPIGateway config entry subscription in proxycfg-glue * Add API gateway to config snapshot validation * Add API gateway to config snapshot clone, leaf, etc. * Subscribe to bound route + cert config entries on bound-api-gateway * Track routes + certs on API gateway config snapshot * Generate DeepCopy() for types used in watch.Map * Watch all active references on api-gateway, unwatch inactive * Track loading of initial bound-api-gateway config entry * Use proper proto package for SDS mapping * Use ResourceReference instead of ServiceName, collect resources * Fix typo, add + remove TODOs * Watch discovery chains for TCPRoute * Add TODO for updating gateway services for api-gateway * make proto * Regenerate deep-copy for proxycfg * Set datacenter on upstream ID from query source * Watch discovery chains for http-route service backends * Add ServiceName getter to HTTP+TCP Service structs * Clean up unwatched discovery chains on API Gateway * Implement watch for ingress leaf certificate * Collect upstreams on http-route + tcp-route updates * Remove unused GatewayServices update handler * Remove unnecessary gateway services logic for API Gateway * Remove outdate TODO * Use .ToIngress where appropriate, including TODO for cleaning up * Cancel before returning error * Remove GatewayServices subscription * Add godoc for handlerAPIGateway functions * Update terminology from Connect => Consul Service Mesh Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690 * Add missing TODO * Remove duplicate switch case * Rerun deep-copy generator * Use correct property on config snapshot * Remove unnecessary leaf cert watch * Clean up based on code review feedback * Note handler properties that are initialized but set elsewhere * Add TODO for moving helper func into structs pkg * Update generated DeepCopy code * gofmt * Begin stubbing for SDS * Start adding tests * Remove second BoundAPIGateway case in glue * TO BE PICKED: fix formatting of str * WIP * Fix merge conflict * Implement HTTP Route to Discovery Chain config entries * Stub out function to create discovery chain * Add discovery chain merging code (#16131) * Test adding TCP and HTTP routes * Add some tests for the synthesizer * Run go mod tidy * Pairing with N8 * Run deep copy * Clean up GatewayChainSynthesizer * Fix missing assignment of BoundAPIGateway topic * Separate out synthesizeChains and toIngressTLS * Fix build errors * Ensure synthesizer skips non-matching routes by protocol * Rebase on N8s work * Generate DeepCopy() for API gateway listener types * Improve variable name * Regenerate DeepCopy() code * Fix linting issue * fix protobuf import * Fix more merge conflict errors * Fix synthesize test * Run deep copy * Add URLRewrite to proto * Update agent/consul/discoverychain/gateway_tcproute.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Remove APIGatewayConfigEntry that was extra * Error out if route kind is unknown * Fix formatting errors in proto --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago