consul

Commit Graph

Author	SHA1	Message	Date
Derek Menteer	4f6da20fe5	Fix multiple issues related to proxycfg health queries. (#17241 ) Fix multiple issues related to proxycfg health queries. 1. The datacenter was not being provided to a proxycfg query, which resulted in bypassing agentless query optimizations and using the normal API instead. 2. The health rpc endpoint would return a zero index when insufficient ACLs were detected. This would result in the agent cache performing an infinite loop of queries in rapid succession without backoff.	2 years ago
Dan Upton	972998203e	controller: deduplicate items in queue (#17168 )	2 years ago
Dan Upton	6e1bc57469	Controller Runtime	2 years ago
Matt Keeler	34915670f2	Register new catalog & mesh protobuf types with the resource registry (#17225 )	2 years ago
Derek Menteer	50ef6a697e	Fix issue with peer stream node cleanup. (#17235 ) Fix issue with peer stream node cleanup. This commit encompasses a few problems that are closely related due to their proximity in the code. 1. The peerstream utilizes node IDs in several locations to determine which nodes / services / checks should be cleaned up or created. While VM deployments with agents will likely always have a node ID, agentless uses synthetic nodes and does not populate the field. This means that for consul-k8s deployments, all services were likely bundled together into the same synthetic node in some code paths (but not all), resulting in strange behavior. The Node.Node field should be used instead as a unique identifier, as it should always be populated. 2. The peerstream cleanup process for unused nodes uses an incorrect query for node deregistration. This query is NOT namespace aware and results in the node (and corresponding services) being deregistered prematurely whenever it has zero default-namespace services and 1+ non-default-namespace services registered on it. This issue is tricky to find due to the incorrect logic mentioned in #1, combined with the fact that the affected services must be co-located on the same node as the currently deregistering service for this to be encountered. 3. The stream tracker did not understand differences between services in different namespaces and could therefore report incorrect numbers. It was updated to utilize the full service name to avoid conflicts and return proper results.	2 years ago
Semir Patel	991a002fcc	resource: List resources by owner (#17190 )	2 years ago
Dan Upton	917afcf3c6	controller: make the `WorkQueue` generic (#16982 )	2 years ago
John Eikenberry	bd76fdeaeb	enable auto-tidy expired issuers in vault (as CA) When using vault as a CA and generating the local signing cert, try to enable the PKI endpoint's auto-tidy feature with it set to tidy expired issuers.	2 years ago
Nathan Coleman	bdef22354b	Use auth context when evaluating service read permissions (#17207 ) Co-authored-by: Blake Covarrubias <1812+blake@users.noreply.github.com>	2 years ago
Poonam Jadhav	ef5d54fd4c	feat: add no-op reporting background routine (#17178 )	2 years ago
Eric Haberkorn	2c0da88ce7	fix panic in `injectSANMatcher` when `tlsContext` is `nil` (#17185 )	2 years ago
Paul Glass	e4a341c88a	Permissive mTLS: Config entry filtering and CLI warnings (#17183 ) This adds filtering for service-defaults: consul config list -filter 'MutualTLSMode == "permissive"'. It adds CLI warnings when the CLI writes a config entry and sees that either service-defaults or proxy-defaults contains MutualTLSMode=permissive, or sees that the mesh config entry contains AllowEnablingPermissiveMutualTLSMode=true.	2 years ago
R.B. Boyer	6b4986907d	peering: ensure that merged central configs of peered upstreams for partitioned downstreams work (#17179 ) Partitioned downstreams with peered upstreams could not properly merge central config info (i.e. proxy-defaults and service-defaults things like mesh gateway modes) if the upstream had an empty DestinationPartition field in Enterprise. Due to data flow, if this setup is done using Consul client agents the field is never empty and thus does not experience the bug. When a service is registered directly to the catalog as is the case for consul-dataplane use this field may be empty and and the internal machinery of the merging function doesn't handle this well. This PR ensures the internal machinery of that function is referentially self-consistent.	2 years ago
Semir Patel	1037bf7f69	Sync .golangci.yml from ENT (#17180 )	2 years ago
John Landa	eded58b62a	Remove artificial ACLTokenMaxTTL limit for configuring acl token expiry (#17066 ) * Remove artificial ACLTokenMaxTTL limit for configuring acl token expiry * Add changelog * Remove test on default MaxTokenTTL * Change to imperitive tense for changelog entry	2 years ago
Semir Patel	9fef1c7f17	Create tombstone on resource `Delete` (#17108 )	2 years ago
Dan Upton	eff5dd1812	resource: owner references must include a uid (#17169 )	2 years ago
Freddy	e02ef16f02	Update HCP bootstrapping to support existing clusters (#16916 ) * Persist HCP management token from server config We want to move away from injecting an initial management token into Consul clusters linked to HCP. The reasoning is that by using a separate class of token we can have more flexibility in terms of allowing HCP's token to co-exist with the user's management token. Down the line we can also more easily adjust the permissions attached to HCP's token to limit it's scope. With these changes, the cloud management token is like the initial management token in that iit has the same global management policy and if it is created it effectively bootstraps the ACL system. * Update SDK and mock HCP server The HCP management token will now be sent in a special field rather than as Consul's "initial management" token configuration. This commit also updates the mock HCP server to more accurately reflect the behavior of the CCM backend. * Refactor HCP bootstrapping logic and add tests We want to allow users to link Consul clusters that already exist to HCP. Existing clusters need care when bootstrapped by HCP, since we do not want to do things like change ACL/TLS settings for a running cluster. Additional changes: * Deconstruct MaybeBootstrap so that it can be tested. The HCP Go SDK requires HTTPS to fetch a token from the Auth URL, even if the backend server is mocked. By pulling the hcp.Client creation out we can modify its TLS configuration in tests while keeping the secure behavior in production code. * Add light validation for data received/loaded. * Sanitize initial_management token from received config, since HCP will only ever use the CloudConfig.MangementToken. * Add changelog entry	2 years ago
John Maguire	391ed069c4	APIGW: Update how status conditions for certificates are handled (#17115 ) * Move status condition for invalid certifcate to reference the listener that is using the certificate * Fix where we set the condition status for listeners and certificate refs, added tests * Add changelog	2 years ago
Semir Patel	5eaeb7b8e5	Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks (#15979 ) * Add MaxEjectionPercent to config entry * Add BaseEjectionTime to config entry * Add MaxEjectionPercent and BaseEjectionTime to protobufs * Add MaxEjectionPercent and BaseEjectionTime to api * Fix integration test breakage * Verify MaxEjectionPercent and BaseEjectionTime in integration test upstream confings * Website docs for MaxEjectionPercent and BaseEjection time * Add `make docs` to browse docs at http://localhost:3000 * Changelog entry * so that is the difference between consul-docker and dev-docker * blah * update proto funcs * update proto --------- Co-authored-by: Maliz <maliheh.monshizadeh@hashicorp.com>	2 years ago
Michael Wilkerson	80b1dbcc7d	fixed aliases for sameness group (sameness_group) (#17161 )	2 years ago
Eric Haberkorn	a87115c598	add acl filter logs (#17143 )	2 years ago
Dan Upton	faae7bb5f2	testing: `RunResourceService` helper (#17068 )	2 years ago
Semir Patel	e7bb8fdf15	Fix or disable pipeline breaking changes that made it into main in last day or so (#17130 ) * Fix straggler from renaming Register->RegisterTypes * somehow a lint failure got through previously * Fix lint-consul-retry errors * adding in fix for success jobs getting skipped. (#17132) * Temporarily disable inmem backend conformance test to get green pipeline * Another test needs disabling --------- Co-authored-by: John Murret <john.murret@hashicorp.com>	2 years ago
Dan Upton	b9c485dcb8	Controller Supervision (#17016 )	2 years ago
John Maguire	e47f3216e5	APIGW Normalize Status Conditions (#16994 ) * normalize status conditions for gateways and routes * Added tests for checking condition status and panic conditions for validating combinations, added dummy code for fsm store * get rid of unneeded gateway condition generator struct * Remove unused file * run go mod tidy * Update tests, add conflicted gateway status * put back removed status for test * Fix linting violation, remove custom conflicted status * Update fsm commands oss * Fix incorrect combination of type/condition/status * cleaning up from PR review * Change "invalidCertificate" to be of accepted status * Move status condition enums into api package * Update gateways controller and generated code * Update conditions in fsm oss tests * run go mod tidy on consul-container module to fix linting * Fix type for gateway endpoint test * go mod tidy from changes to api * go mod tidy on troubleshoot * Fix route conflicted reason * fix route conflict reason rename * Fix text for gateway conflicted status * Add valid certificate ref condition setting * Revert change to resolved refs to be handled in future PR	2 years ago
Michael Wilkerson	001d540afc	Add sameness group field to prepared queries (#17089 ) * added method for converting SamenessGroupConfigEntry - added new method `ToQueryFailoverTargets` for converting a SamenessGroupConfigEntry's members to a list of QueryFailoverTargets - renamed `ToFailoverTargets` ToServiceResolverFailoverTargets to distinguish it from `ToQueryFailoverTargets` * Added SamenessGroup to PreparedQuery - exposed Service.Partition to API when defining a prepared query - added a method for determining if a QueryFailoverOptions is empty - This will be useful for validation - added unit tests * added method for retrieving a SamenessGroup to state store * added logic for using PQ with SamenessGroup - added branching path for SamenessGroup handling in execute. It will be handled separate from the normal PQ case - added a new interface so that the `GetSamenessGroupFailoverTargets` can be properly tested - separated the execute logic into a `targetSelector` function so that it can be used for both failover and sameness group PQs - split OSS only methods into new PQ OSS files - added validation that `samenessGroup` is an enterprise only feature * added documentation for PQ SamenessGroup	2 years ago
Derek Menteer	a33b224a55	Fix virtual services being included in intention topology as downstreams. (#17099 )	2 years ago
Semir Patel	46816071df	De-scope tenenacy requirements to OSS only for now. (#17087 ) Partition and namespace must be "default" Peername must be "local"	2 years ago
Kyle Havlovitz	6d01d07cf8	Include virtual services from discovery chain in intention topology (#16862 )	2 years ago
Kyle Havlovitz	d5277af70d	Add manual virtual IP support to state store (#16815 )	2 years ago
Eric Haberkorn	53cdda8d17	Fix a bug with disco chain config entry fetching (#17078 ) Before this change, we were not fetching service resolvers (and therefore service defaults) configuration entries for services on members of sameness groups.	2 years ago
Semir Patel	53f49b2fa1	Enforce operator:write acl on `WriteStatus` endpoint (#17019 )	2 years ago
Eric Haberkorn	b1fae05983	Add sameness groups to service intentions. (#17064 )	2 years ago
hashicorp-copywrite[bot]	9f81fc01e9	[COMPLIANCE] Add Copyright and License Headers (#16854 ) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>	2 years ago
Paul Glass	f4406e69b9	[NET-3091] Update service intentions to support jwt provider references (#17037 ) * [NET-3090] Add new JWT provider config entry * Add initial test cases * update validations for jwt-provider config entry fields * more validation * start improving tests * more tests * Normalize * Improve tests and move validate fns * usage test update * Add split between ent and oss for partitions * fix lint issues * Added retry backoff, fixed tests, removed unused defaults * take into account default partitions * use countTrue and add aliases * omit audiences if empty * fix failing tests * add omit-entry * Add JWT intentions * generate proto * fix deep copy issues * remove extra field * added some tests * more tests * add validation for creating existing jwt * fix nil issue * More tests, fix conflicts and improve memdb call * fix namespace * add aliases * consolidate errors, skip duplicate memdb calls * reworked iteration over config entries * logic improvements from review --------- Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>	2 years ago
Paul Glass	ac200cfec8	[NET-3090] Add new JWT provider config entry (#17036 ) * [NET-3090] Add new JWT provider config entry * Add initial test cases * update validations for jwt-provider config entry fields * more validation * start improving tests * more tests * Normalize * Improve tests and move validate fns * usage test update * Add split between ent and oss for partitions * fix lint issues * Added retry backoff, fixed tests, removed unused defaults * take into account default partitions * use countTrue and add aliases * omit audiences if empty * fix failing tests * add omit-entry * update copyright headers ids --------- Co-authored-by: Ronald Ekambi <ronekambi@gmail.com> Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>	2 years ago
Paul Glass	77ecff3209	Permissive mTLS (#17035 ) This implements permissive mTLS , which allows toggling services into "permissive" mTLS mode. Permissive mTLS mode allows incoming "non Consul-mTLS" traffic to be forward unmodified to the application. * Update service-defaults and proxy-defaults config entries with a MutualTLSMode field * Update the mesh config entry with an AllowEnablingPermissiveMutualTLS field and implement the necessary validation. AllowEnablingPermissiveMutualTLS must be true to allow changing to MutualTLSMode=permissive, but this does not require that all proxy-defaults and service-defaults are currently in strict mode. * Update xDS listener config to add a "permissive filter chain" when MutualTLSMode=permissive for a particular service. The permissive filter chain matches incoming traffic by the destination port. If the destination port matches the service port from the catalog, then no mTLS is required and the traffic sent is forwarded unmodified to the application.	2 years ago
R.B. Boyer	d07aac8d7e	Revert "cache: refactor agent cache fetching to prevent unnecessary f… (#16818 ) (#17046 ) Revert "cache: refactor agent cache fetching to prevent unnecessary fetches on error (#14956)" Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com>	2 years ago
John Murret	2cefa8d9bd	ci: remove test-integrations CircleCI workflow (#16928 ) * remove all CircleCI files * remove references to CircleCI * remove more references to CircleCI * pin golangci-lint to v1.51.1 instead of v1.51	2 years ago
Luke Kysow	46212cc570	Don't send updates twice (#16999 )	2 years ago
Poonam Jadhav	5d7a7ff041	feat: set up reporting agent (#16991 )	2 years ago
Dan Upton	a37a441991	server: wire up in-process Resource Service (#16978 )	2 years ago
Semir Patel	2f7d591702	Tenancy wildcard validaton for `Write`, `Read`, and `Delete` endpoints (#17004 )	2 years ago
Derek Menteer	87324c9ec8	Add PrioritizeByLocality to config entries. (#17007 ) This commit adds the PrioritizeByLocality field to both proxy-config and service-resolver config entries for locality-aware routing. The field is currently intended for enterprise only, and will be used to enable prioritization of service-mesh connections to services based on geographical region / zone.	2 years ago
Michael Wilkerson	0dd4ea2033	* added Sameness Group to proto files (#16998 ) - added Sameness Group to config entries - added Sameness Group to subscriptions * generated proto files * added Sameness Group events to the state store - added test cases * Refactored health RPC Client - moved code that is common to rpcclient under rpcclient common.go. This will help set us up to support future RPC clients * Refactored proxycfg glue views - Moved views to rpcclient config entry. This will allow us to reuse this code for a config entry client * added config entry RPC Client - Copied most of the testing code from rpcclient/health * hooked up new rpcclient in agent * fixed documentation and comments for clarity	2 years ago
Dhia Ayachi	79d4040b6c	add IP rate limiting config update (#16997 ) * add IP rate limiting config update * fix review comments	2 years ago
Semir Patel	79b30476e0	Enforce Owner rules in `Write` endpoint (#16983 )	2 years ago
Semir Patel	8611ec56f3	Fix delete when uid not provided (#16996 )	2 years ago
Eric Haberkorn	44b39240a8	move enterprise test cases out of open source (#16985 )	2 years ago
Semir Patel	b8c9e133be	Add mutate hook to `Write` endpoint (#16958 )	2 years ago
Semir Patel	3b83c7ee9a	Enforce ACLs on resource `Write` and `Delete` endpoints (#16956 )	2 years ago
Dhia Ayachi	b85a149eaf	Memdb Txn Commit race condition fix (#16871 ) * Add a test to reproduce the race condition * Fix race condition by publishing the event after the commit and adding a lock to prevent out of order events. * split publish to generate the list of events before committing the transaction. * add changelog * remove extra func * Apply suggestions from code review Co-authored-by: Dan Upton <daniel@floppy.co> * add comment to explain test --------- Co-authored-by: Dan Upton <daniel@floppy.co>	2 years ago
Poonam Jadhav	8255cc97f5	feat: add reporting config with reload (#16890 )	2 years ago
Dan Upton	d595e6ade9	resource: `WriteStatus` endpoint (#16886 )	2 years ago
Derek Menteer	1bcaeabfc3	Remove deprecated service-defaults upstream behavior. (#16957 ) Prior to this change, peer services would be targeted by service-default overrides as long as the new `peer` field was not found in the config entry. This commit removes that deprecated backwards-compatibility behavior. Now it is necessary to specify the `peer` field in order for upstream overrides to apply to a peer upstream.	2 years ago
Semir Patel	317240fca7	Resource validation hook for `Write` endpoint (#16950 )	2 years ago
Semir Patel	686f49346c	Check acls on resource `Read`, `List`, and `WatchList` (#16842 )	2 years ago
John Maguire	92be8bd762	APIGW: Routes with duplicate parents should be invalid (#16926 ) * ensure route parents are unique when creating an http route * Ensure tcp route parents are unique * Added unit tests	2 years ago
John Eikenberry	97173725b7	log warning about certificate expiring sooner and with more details The old setting of 24 hours was not enough time to deal with an expiring certificates. This change ups it to 28 days OR 40% of the full cert duration, whichever is shorter. It also adds details to the log message to indicate which certificate it is logging about and a suggested action.	2 years ago
Chris Thain	175bb1a303	Wasm Envoy HTTP extension (#16877 )	2 years ago
Semir Patel	1794484298	Resource `Delete` endpoint (#16756 )	2 years ago
Dan Upton	4fa2537b3b	Resource `Write` endpoint (#16786 )	2 years ago
Dan Upton	671d5825ca	Raft storage backend (#16619 )	2 years ago
cskh	a319953576	docs: add envoy to the proxycfg diagram (#16834 ) * docs: add envoy to the proxycfg diagram	2 years ago
Freddy	f6de5ff635	Allow dialer to re-establish terminated peering (#16776 ) Currently, if an acceptor peer deletes a peering the dialer's peering will eventually get to a "terminated" state. If the two clusters need to be re-peered the acceptor will re-generate the token but the dialer will encounter this error on the call to establish: "failed to get addresses to dial peer: failed to refresh peer server addresses, will continue to use initial addresses: there is no active peering for "<<<ID>>>"" This is because in `exchangeSecret().GetDialAddresses()` we will get an error if fetching addresses for an inactive peering. The peering shows up as inactive at this point because of the existing terminated state. Rather than checking whether a peering is active we can instead check whether it was deleted. This way users do not need to delete terminated peerings in the dialing cluster before re-establishing them.	2 years ago
Chris S. Kim	a5397b1f23	Connect CA Primary Provider refactor (#16749 ) * Rename Intermediate cert references to LeafSigningCert Within the Consul CA subsystem, the term "Intermediate" is confusing because the meaning changes depending on provider and datacenter (primary vs secondary). For example, when using the Consul CA the "ActiveIntermediate" may return the root certificate in a primary datacenter. At a high level, we are interested in knowing which CA is responsible for signing leaf certs, regardless of its position in a certificate chain. This rename makes the intent clearer. * Move provider state check earlier * Remove calls to GenerateLeafSigningCert GenerateLeafSigningCert (formerly known as GenerateIntermediate) is vestigial in non-Vault providers, as it simply returns the root certificate in primary datacenters. By folding Vault's intermediate cert logic into `GenerateRoot` we can encapsulate the intermediate cert handling within `newCARoot`. * Move GenerateLeafSigningCert out of PrimaryProvidder Now that the Vault Provider calls GenerateLeafSigningCert within GenerateRoot, we can remove the method from all other providers that never used it in a meaningful way. * Add test for IntermediatePEM * Rename GenerateRoot to GenerateCAChain "Root" was being overloaded in the Consul CA context, as different providers and configs resulted in a single root certificate or a chain originating from an external trusted CA. Since the Vault provider also generates intermediates, it seems more accurate to call this a CAChain.	2 years ago
Eric Haberkorn	a6d69adcf5	Add default resolvers to disco chains based on the default sameness group (#16837 )	2 years ago
Derek Menteer	8d40cf9858	Add sameness-group to exported-services config entries (#16836 ) This PR adds the sameness-group field to exported-service config entries, which allows for services to be exported to multiple destination partitions / peers easily.	2 years ago
Dan Upton	651549c97d	storage: fix resource leak in Watch (#16817 )	2 years ago
Eric Haberkorn	0d1d2fc4c9	add order by locality failover to Consul enterprise (#16791 )	2 years ago
Ronald	b64674623e	Copyright headers for missing files/folders (#16708 ) * copyright headers for agent folder	2 years ago
Ronald	94ec4eb2f4	copyright headers for agent folder (#16704 ) * copyright headers for agent folder * Ignore test data files * fix proto files and remove headers in agent/uiserver folder * ignore deep-copy files	2 years ago
John Maguire	c833464daf	Update normalization of route refs (#16789 ) * Use merge of enterprise meta's rather than new custom method * Add merge logic for tcp routes * Add changelog * Normalize certificate refs on gateways * Fix infinite call loop * Explicitly call enterprise meta	2 years ago
Michael Wilkerson	e5d58c59c9	changes to support new PQ enterprise fields (#16793 )	2 years ago
Semir Patel	440f11203f	Resource service List(..) endpoint (#16753 )	2 years ago
Dhia Ayachi	10df4d83aa	add ip rate limiter controller OSS parts (#16790 )	2 years ago
Kyle Havlovitz	42c5b29713	Allocate virtual ip for resolver/router/splitter config entries (#16760 )	2 years ago
Semir Patel	032aba3175	WatchList(..) endpoint for the resource service (#16726 )	2 years ago
John Maguire	351bdc3c0d	Fix struct tags for TCPService enterprise meta (#16781 ) * Fix struct tags for TCPService enterprise meta * Add changelog	2 years ago
Semir Patel	3415689eb6	Read(...) endpoint for the resource service (#16655 )	2 years ago
Derek Menteer	2236975011	Change partition for peers in discovery chain targets (#16769 ) This commit swaps the partition field to the local partition for discovery chains targeting peers. Prior to this change, peer upstreams would always use a value of default regardless of which partition they exist in. This caused several issues in xds / proxycfg because of id mismatches. Some prior fixes were made to deal with one-off id mismatches that this PR also cleans up, since they are no longer needed.	2 years ago
John Eikenberry	0b1dc4ec36	tests instantiating clients w/o shutting down (#16755 ) noticed via their port still in use messages.	2 years ago
Poonam Jadhav	3df271959c	fix: remove unused tenancy category from rate limit spec (#16740 )	2 years ago
Dhia Ayachi	3ba0eb5074	delete config when nil (#16690 ) * delete config when nil * fix mock interface implementation * fix handler test to use the right assertion * extract DeleteConfig as a separate API. * fix mock limiter implementation to satisfy the new interface * fix failing tests * add test comments	2 years ago
Eric Haberkorn	495ad4c7ef	add enterprise xds tests (#16738 )	2 years ago
Eric Haberkorn	3c5c53aa80	fix bug where pqs that failover to a cluster peer dont un-fail over (#16729 )	2 years ago
cskh	7f6f6891f7	fix: gracefully fail on invalid port number (#16721 )	2 years ago
John Maguire	8dd1d73874	Remove unused are hosts set check (#16691 ) * Remove unused are hosts set check * Remove all traces of unused 'AreHostsSet' parameter * Remove unused Hosts attribute * Remove commented out use of snap.APIGateway.Hosts	2 years ago
Nitya Dhanushkodi	b9bd2c3780	peering: peering partition failover fixes (#16673 ) add local source partition for peered upstreams	2 years ago
John Maguire	1ef9f4dade	Fix route subscription when using namespaces (#16677 ) * Fix route subscription when using namespaces * Update changelog * Fix changelog entry to reference that the bug was enterprise only	2 years ago
Melisa Griffin	606f8fbbab	Adds check to verify that the API Gateway is being created with at least one listener	2 years ago
Poonam Jadhav	9c64731a56	feat: add category annotation to RPC and gRPC methods (#16646 )	2 years ago
Eric Haberkorn	7477f52a16	add sameness groups to discovery chains (#16671 )	2 years ago
Andrew Stucki	501b87fd31	[API Gateway] Fix invalid cluster causing gateway programming delay (#16661 ) * Add test for http routes * Add fix * Fix tests * Add changelog entry * Refactor and fix flaky tests	2 years ago
Eric Haberkorn	eaa39f4ef5	add sameness group support to service resolver failover and redirects (#16664 )	2 years ago
Eric Haberkorn	57e034b746	fix confusing spiffe ids in golden tests (#16643 )	2 years ago
wangxinyi7	152c75349e	net 2731 ip config entry OSS version (#16642 ) * ip config entry * name changing * move to ent * ent version * renaming * change format * renaming * refactor * add default values	2 years ago
John Maguire	ff5887a99e	Update e2e tests for namespaces (#16627 ) * Refactored "NewGatewayService" to handle namespaces, fixed TestHTTPRouteFlattening test * Fixed existing http_route tests for namespacing * Squash aclEnterpriseMeta for ResourceRefs and HTTPServices, accept namespace for creating connect services and regular services * Use require instead of assert after creating namespaces in http_route_tests * Refactor NewConnectService and NewGatewayService functions to use cfg objects to reduce number of method args * Rename field on SidecarConfig in tests from `SidecarServiceName` to `Name` to avoid stutter	2 years ago
Freddy	724b752ca7	Backport ENT-4704 (#16612 )	2 years ago
Derek Menteer	8f75d99299	Fix issue with trust bundle read ACL check. (#16630 ) This commit fixes an issue where trust bundles could not be read by services in a non-default namespace, unless they had excessive ACL permissions given to them. Prior to this change, `service:write` was required in the default namespace in order to read the trust bundle. Now, `service:write` to a service in any namespace is sufficient.	2 years ago
Chris S. Kim	d5677e5680	Preserve CARoots when updating Vault CA configuration (#16592 ) If a CA config update did not cause a root change, the codepath would return early and skip some steps which preserve its intermediate certificates and signing key ID. This commit re-orders some code and prevents updates from generating new intermediate certificates.	2 years ago
Derek Menteer	f2902e6608	Add sameness-group configuration entry. (#16608 ) This commit adds a sameness-group config entry to the API and structs packages. It includes some validation logic and a new memdb index that tracks the default sameness-group for each partition. Sameness groups will simplify the effort of managing failovers / intentions / exports for peers and partitions. Note that this change purely to introduce the configuration entry and does not include the full functionality of sameness-groups.	2 years ago
Ashvitha	f95ffe0355	Allow HCP metrics collection for Envoy proxies Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com> Add a new envoy flag: "envoy_hcp_metrics_bind_socket_dir", a directory where a unix socket will be created with the name `<namespace>_<proxy_id>.sock` to forward Envoy metrics. If set, this will configure: - In bootstrap configuration a local stats_sink and static cluster. These will forward metrics to a loopback listener sent over xDS. - A dynamic listener listening at the socket path that the previously defined static cluster is sending metrics to. - A dynamic cluster that will forward traffic received at this listener to the hcp-metrics-collector service. Reasons for having a static cluster pointing at a dynamic listener: - We want to secure the metrics stream using TLS, but the stats sink can only be defined in bootstrap config. With dynamic listeners/clusters we can use the proxy's leaf certificate issued by the Connect CA, which isn't available at bootstrap time. - We want to intelligently route to the HCP collector. Configuring its addreess at bootstrap time limits our flexibility routing-wise. More on this below. Reasons for defining the collector as an upstream in `proxycfg`: - The HCP collector will be deployed as a mesh service. - Certificate management is taken care of, as mentioned above. - Service discovery and routing logic is automatically taken care of, meaning that no code changes are required in the xds package. - Custom routing rules can be added for the collector using discovery chain config entries. Initially the collector is expected to be deployed to each admin partition, but in the future could be deployed centrally in the default partition. These config entries could even be managed by HCP itself.	2 years ago
Eric Haberkorn	e298f506a5	Add Peer Locality to Discovery Chains (#16588 ) Add peer locality to discovery chains	2 years ago
Eric Haberkorn	57e2493415	allow setting locality on services and nodes (#16581 )	2 years ago
Semir Patel	176945aa86	GRPC stub for the ResourceService (#16528 )	2 years ago
Andrew Stucki	040647e0ba	auto-updated agent/uiserver/dist/ from commit `63204b518` (#16587 ) Co-authored-by: hc-github-team-consul-core <github-team-consul-core@hashicorp.com>	2 years ago
Eric Haberkorn	89de91b263	fix bug that can lead to peering service deletes impacting the state of local services (#16570 )	2 years ago
Eric Haberkorn	dbaf8bf49c	add agent locality and replicate it across peer streams (#16522 )	2 years ago
John Eikenberry	f5641ffccc	support vault auth config for alicloud ca provider Add support for using existing vault auto-auth configurations as the provider configuration when using Vault's CA provider with AliCloud. AliCloud requires 2 extra fields to enable it to use STS (it's preferred auth setup). Our vault-plugin-auth-alicloud package contained a method to help generate them as they require you to make an http call to a faked endpoint proxy to get them (url and headers base64 encoded).	2 years ago
Melisa Griffin	fc232326a0	NET-2904 Fixes API Gateway Route Service Weight Division Error	2 years ago
Melisa Griffin	129eca8fdb	NET-2903 Normalize weight for http routes (#16512 ) * NET-2903 Normalize weight for http routes * Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>	2 years ago
R.B. Boyer	9a485cdb49	proxycfg: ensure that an irrecoverable error in proxycfg closes the xds session and triggers a replacement proxycfg watcher (#16497 ) Receiving an "acl not found" error from an RPC in the agent cache and the streaming/event components will cause any request loops to cease under the assumption that they will never work again if the token was destroyed. This prevents log spam (#14144, #9738). Unfortunately due to things like: - authz requests going to stale servers that may not have witnessed the token creation yet - authz requests in a secondary datacenter happening before the tokens get replicated to that datacenter - authz requests from a primary TO a secondary datacenter happening before the tokens get replicated to that datacenter The caller will get an "acl not found" before the token exists, rather than just after. The machinery added above in the linked PRs will kick in and prevent the request loop from looping around again once the tokens actually exist. For `consul-dataplane` usages, where xDS is served by the Consul servers rather than the clients ultimately this is not a problem because in that scenario the `agent/proxycfg` machinery is on-demand and launched by a new xDS stream needing data for a specific service in the catalog. If the watching goroutines are terminated it ripples down and terminates the xDS stream, which CDP will eventually re-establish and restart everything. For Consul client usages, the `agent/proxycfg` machinery is ahead-of-time launched at service registration time (called "local" in some of the proxycfg machinery) so when the xDS stream comes in the data is already ready to go. If the watching goroutines terminate it should terminate the xDS stream, but there's no mechanism to re-spawn the watching goroutines. If the xDS stream reconnects it will see no `ConfigSnapshot` and will not get one again until the client agent is restarted, or the service is re-registered with something changed in it. This PR fixes a few things in the machinery: - there was an inadvertent deadlock in fetching snapshot from the proxycfg machinery by xDS, such that when the watching goroutine terminated the snapshots would never be fetched. This caused some of the xDS machinery to get indefinitely paused and not finish the teardown properly. - Every 30s we now attempt to re-insert all locally registered services into the proxycfg machinery. - When services are re-inserted into the proxycfg machinery we special case "dead" ones such that we unilaterally replace them rather that doing that conditionally.	2 years ago
John Eikenberry	56ffee6d42	add provider ca support for approle auth-method Adds support for the approle auth-method. Only handles using the approle role/secret to auth and it doesn't support the agent's extra management configuration options (wrap and delete after read) as they are not required as part of the auth (ie. they are vault agent things).	2 years ago
Andrew Stucki	cc0765b87d	Fix resolution of service resolvers with subsets for external upstreams (#16499 ) * Fix resolution of service resolvers with subsets for external upstreams * Add tests * Add changelog entry * Update view filter logic	2 years ago
Eric Haberkorn	5f81662066	Add support for failover policies (#16505 )	2 years ago
Andrew Stucki	5deffbd95b	Fix issue where terminating gateway service resolvers weren't properly cleaned up (#16498 ) * Fix issue where terminating gateway service resolvers weren't properly cleaned up * Add integration test for cleaning up resolvers * Add changelog entry * Use state test and drop integration test	2 years ago
Andrew Stucki	4b661d1e0c	Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable (#16495 ) * Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable * Regenerate golden files * Add RequestTimeout field * Add changelog entry	2 years ago
John Eikenberry	e8eec1fa80	add provider ca auth support for kubernetes Adds support for Kubernetes jwt/token file based auth. Only needs to read the file and save the contents as the jwt/token.	2 years ago
John Eikenberry	4211069080	add provider ca support for jwt file base auth Adds support for a jwt token in a file. Simply reads the file and sends the read in jwt along to the vault login. It also supports a legacy mode with the jwt string being passed directly. In which case the path is made optional.	2 years ago
Chris S. Kim	321439f5a7	Speed up test by registering services concurrently (#16509 )	2 years ago
John Eikenberry	4f2d9a91e5	add provider ca auth-method support for azure Does the required dance with the local HTTP endpoint to get the required data for the jwt based auth setup in Azure. Keeps support for 'legacy' mode where all login data is passed on via the auth methods parameters. Refactored check for hardcoded /login fields.	2 years ago
Dan Upton	73b9b407ba	grpc: fix data race in balancer registration (#16229 ) Registering gRPC balancers is thread-unsafe because they are stored in a global map variable that is accessed without holding a lock. Therefore, it's expected that balancers are registered _once_ at the beginning of your program (e.g. in a package `init` function) and certainly not after you've started dialing connections, etc. > NOTE: this function must only be called during initialization time > (i.e. in an init() function), and is not thread-safe. While this is fine for us in production, it's challenging for tests that spin up multiple agents in-memory. We currently register a balancer per- agent which holds agent-specific state that cannot safely be shared. This commit introduces our own registry that _is_ thread-safe, and implements the Builder interface such that we can call gRPC's `Register` method once, on start-up. It uses the same pattern as our resolver registry where we use the dial target's host (aka "authority"), which is unique per-agent, to determine which builder to use.	2 years ago
Andrew Stucki	801a17329e	Fix attempt for test fail panics in xDS (#16319 ) * Fix attempt for test fail panics in xDS * switch to a mutex pointer	2 years ago
Chris S. Kim	a518893685	Fix various flaky tests (#16396 )	2 years ago
Eric Haberkorn	595131fca9	Refactor the disco chain -> xds logic (#16392 )	2 years ago
Paul Banks	8ac211b427	Correct WAL metrics registrations (#16388 )	2 years ago
Dhia Ayachi	ae9c228967	Rate limiter/add ip prefix (#16342 ) * add support for prefixes in the config tree * fix to use default config when the prefix have no config	2 years ago
Andrew Stucki	641737f32b	[API Gateway] Fix infinite loop in controller and binding non-accepted routes and gateways (#16377 )	2 years ago
Andrew Stucki	0972697661	[API Gateway] Various fixes for Config Entry fields (#16347 ) * [API Gateway] Various fixes for Config Entry fields * simplify logic per PR review	2 years ago
Andrew Stucki	18e2ee77ca	[API Gateway] Fix targeting service splitters in HTTPRoutes (#16350 ) * [API Gateway] Fix targeting service splitters in HTTPRoutes * Fix test description	2 years ago
Andrew Stucki	823fc821fa	[API Gateway] Turn down controller log levels (#16348 )	2 years ago
Derek Menteer	ad865f549b	Fix issue with peer services incorrectly appearing as connect-enabled. (#16339 ) Prior to this commit, all peer services were transmitted as connect-enabled as long as a one or more mesh-gateways were healthy. With this change, there is now a difference between typical services and connect services transmitted via peering. A service will be reported as "connect-enabled" as long as any of these conditions are met: 1. a connect-proxy sidecar is registered for the service name. 2. a connect-native instance of the service is registered. 3. a service resolver / splitter / router is registered for the service name. 4. a terminating gateway has registered the service.	2 years ago
Andrew Stucki	7f9ec78932	[API Gateway] Validate listener name is not empty (#16340 ) * [API Gateway] Validate listener name is not empty * Update docstrings and test	2 years ago
cskh	8e5942f5ca	fix: add tls config to unix socket when https is used (#16301 ) * fix: add tls config to unix socket when https is used * unit test and changelog	2 years ago
Andrew Stucki	4607b535be	Fix HTTPRoute and TCPRoute expectation for enterprise metadata (#16322 )	2 years ago
Andrew Stucki	15d2684ecc	Normalize all API Gateway references (#16316 )	2 years ago
Matt Keeler	085c0addc0	Protobuf Refactoring for Multi-Module Cleanliness (#16302 ) Protobuf Refactoring for Multi-Module Cleanliness This commit includes the following: Moves all packages that were within proto/ to proto/private Rewrites imports to account for the packages being moved Adds in buf.work.yaml to enable buf workspaces Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes) Why: In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage. There were some recent changes to have our own ratelimiting annotations. The two combined were not working when I was trying to use them together (attempting to rebase another branch) Buf workspaces should be the solution to the problem Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root. This resulted in proto file name conflicts in the Go global protobuf type registry. The solution to that was to add in a private/ directory into the path within the proto/ directory. That then required rewriting all the imports. Is this safe? AFAICT yes The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc) Other than imports, there were no changes to any generated code as a result of this.	2 years ago
Dan Stough	f1436109ea	[OSS] security: update go to 1.20.1 (#16263 ) * security: update go to 1.20.1	2 years ago
Andrew Stucki	58801cc8aa	Add stricter validation and some normalization code for API Gateway ConfigEntries (#16304 ) * Add stricter validation and some normalization code for API Gateway ConfigEntries	2 years ago
Andrew Stucki	ee99d5c3a0	Fix panicky xDS test flakes (#16305 ) * Add defensive guard to make some tests less flaky and panic less * Do the actual fix	2 years ago
Andrew Stucki	e4a992c581	Fix hostname alignment checks for HTTPRoutes (#16300 ) * Fix hostname alignment checks for HTTPRoutes	2 years ago
Andrew Stucki	b3ddd4d24e	Inline API Gateway TLS cert code (#16295 ) * Include secret type when building resources from config snapshot * First pass at generating envoy secrets from api-gateway snapshot * Update comments for xDS update order * Add secret type + corresponding golden files to existing tests * Initialize test helpers for testing api-gateway resource generation * Generate golden files for new api-gateway xDS resource test * Support ADS for TLS certificates on api-gateway * Configure TLS on api-gateway listeners * Inline TLS cert code * update tests * Add SNI support so we can have multiple certificates * Remove commented out section from helper * regen deep-copy * Add tcp tls test --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>	2 years ago
Nitya Dhanushkodi	8dab825c36	troubleshoot: fixes and updated messages (#16294 )	2 years ago
Thomas Eckert	2460ac99c9	API Gateway Envoy Golden Listener Tests (#16221 ) * Simple API Gateway e2e test for tcp routes * Drop DNSSans since we don't front the Gateway with a leaf cert * WIP listener tests for api-gateway * Return early if no routes * Add back in leaf cert to testing * Fix merge conflicts * Re-add kind to setup * Fix iteration over listener upstreams * New tcp listener test * Add tests for API Gateway with TCP and HTTP routes * Move zero-route check back * Drop generateIngressDNSSANs * Check for chains not routes --------- Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
Derek Menteer	30112288c8	Fix mesh gateways incorrectly matching peer locality. (#16257 ) Fix mesh gateways incorrectly matching peer locality. This fixes an issue where local mesh gateways use an incorrect address when attempting to forward traffic to a peered datacenter. Prior to this change it would use the lan address instead of the wan if the locality matched. This should never be done for peering, since we must route all traffic through the remote mesh gateway.	2 years ago
Nathan Coleman	514fb25a6f	Fix infinite recursion in inline-certificate config entry (#16276 ) * Fix infinite recursion on InlineCertificateConfigEntry GetNamespace() + GetMeta() were calling themselves. This change also simplifies by removing nil-checking to match pre-existing config entries Co-Authored-By: Andrew Stucki <3577250+andrewstucki@users.noreply.github.com> * Add tests for inline-certificate * Add alias for private key field on inline-certificate * Use valid certificate + private key for inline-certificate tests --------- Co-authored-by: Andrew Stucki <3577250+andrewstucki@users.noreply.github.com>	2 years ago
Derek Menteer	6599a9be1d	Fix nil-pointer panics from proxycfg package. (#16277 ) Prior to this PR, servers / agents would panic and crash if an ingress or api gateway were configured to use a discovery chain that both: 1. Referenced a peered service 2. Had a mesh gateway mode of local This could occur, because code for handling upstream watches was shared between both connect-proxy and the gateways. As a short-term fix, this PR ensures that the maps are always initialized for these gateway services. This PR also wraps the proxycfg execution and service registration calls with recover statements to ensure that future issues like this do not put the server into an unrecoverable state.	2 years ago
Andrew Stucki	9bb0ecfc18	[API Gateway] Add integration test for HTTP routes (#16236 ) * [API Gateway] Add integration test for conflicted TCP listeners * [API Gateway] Update simple test to leverage intentions and multiple listeners * Fix broken unit test * [API Gateway] Add integration test for HTTP routes	2 years ago
Semir Patel	8979e64a94	Bump x/time to 0.3.0 and fix related breakage linked to RPCRateLimit (#16241 ) * Bump x/time to 0.3.0 and fix related breakage linked to RPCRateLimit initialization * Apply limitVal(...) to other rate.Limit config fields	2 years ago
Andrew Stucki	8ff2974dbe	[API Gateway] Update simple test to leverage intentions and multiple listeners (#16228 ) * [API Gateway] Add integration test for conflicted TCP listeners * [API Gateway] Update simple test to leverage intentions and multiple listeners * Fix broken unit test * PR suggestions	2 years ago
Andrew Stucki	4c848a554d	Fix missing references to enterprise metadata (#16237 )	2 years ago
Andrew Stucki	318ba215ab	[API Gateway] Add integration test for conflicted TCP listeners (#16225 )	2 years ago
Derek Menteer	4f2ce60654	Fix peering acceptors in secondary datacenters. (#16230 ) Prior to this commit, secondary datacenters could not be initialized as peering acceptors if ACLs were enabled. This is due to the fact that internal server-to-server API calls would fail because the management token was not generated. This PR makes it so that both primary and secondary datacenters generate their own management token whenever a leader is elected in their respective clusters.	2 years ago
Andrew Stucki	3b9c569561	Simple API Gateway e2e test for tcp routes (#16222 ) * Simple API Gateway e2e test for tcp routes * Drop DNSSans since we don't front the Gateway with a leaf cert	2 years ago
skpratt	db2bd404bf	Synthesize anonymous token pre-bootstrap when needed (#16200 ) * add bootstrapping detail for acl errors * error detail improvements * update acl bootstrapping test coverage * update namespace errors * update test coverage * consolidate error message code and update changelog * synthesize anonymous token * Update token language to distinguish Accessor and Secret ID usage (#16044) * remove legacy tokens * remove lingering legacy token references from docs * update language and naming for token secrets and accessor IDs * updates all tokenID references to clarify accessorID * remove token type references and lookup tokens by accessorID index * remove unnecessary constants * replace additional tokenID param names * Add warning info for deprecated -id parameter Co-authored-by: Paul Glass <pglass@hashicorp.com> * Update field comment Co-authored-by: Paul Glass <pglass@hashicorp.com> --------- Co-authored-by: Paul Glass <pglass@hashicorp.com> * revert naming change * add testing * revert naming change --------- Co-authored-by: Paul Glass <pglass@hashicorp.com>	2 years ago
Thomas Eckert	e81a0c2855	API Gateway to Ingress Gateway Snapshot Translation and Routes to Virtual Routers and Splitters (#16127 ) * Stub proxycfg handler for API gateway * Add Service Kind constants/handling for API Gateway * Begin stubbing for SDS * Add new Secret type to xDS order of operations * Continue stubbing of SDS * Iterate on proxycfg handler for API gateway * Handle BoundAPIGateway config entry subscription in proxycfg-glue * Add API gateway to config snapshot validation * Add API gateway to config snapshot clone, leaf, etc. * Subscribe to bound route + cert config entries on bound-api-gateway * Track routes + certs on API gateway config snapshot * Generate DeepCopy() for types used in watch.Map * Watch all active references on api-gateway, unwatch inactive * Track loading of initial bound-api-gateway config entry * Use proper proto package for SDS mapping * Use ResourceReference instead of ServiceName, collect resources * Fix typo, add + remove TODOs * Watch discovery chains for TCPRoute * Add TODO for updating gateway services for api-gateway * make proto * Regenerate deep-copy for proxycfg * Set datacenter on upstream ID from query source * Watch discovery chains for http-route service backends * Add ServiceName getter to HTTP+TCP Service structs * Clean up unwatched discovery chains on API Gateway * Implement watch for ingress leaf certificate * Collect upstreams on http-route + tcp-route updates * Remove unused GatewayServices update handler * Remove unnecessary gateway services logic for API Gateway * Remove outdate TODO * Use .ToIngress where appropriate, including TODO for cleaning up * Cancel before returning error * Remove GatewayServices subscription * Add godoc for handlerAPIGateway functions * Update terminology from Connect => Consul Service Mesh Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690 * Add missing TODO * Remove duplicate switch case * Rerun deep-copy generator * Use correct property on config snapshot * Remove unnecessary leaf cert watch * Clean up based on code review feedback * Note handler properties that are initialized but set elsewhere * Add TODO for moving helper func into structs pkg * Update generated DeepCopy code * gofmt * Begin stubbing for SDS * Start adding tests * Remove second BoundAPIGateway case in glue * TO BE PICKED: fix formatting of str * WIP * Fix merge conflict * Implement HTTP Route to Discovery Chain config entries * Stub out function to create discovery chain * Add discovery chain merging code (#16131) * Test adding TCP and HTTP routes * Add some tests for the synthesizer * Run go mod tidy * Pairing with N8 * Run deep copy * Clean up GatewayChainSynthesizer * Fix missing assignment of BoundAPIGateway topic * Separate out synthesizeChains and toIngressTLS * Fix build errors * Ensure synthesizer skips non-matching routes by protocol * Rebase on N8s work * Generate DeepCopy() for API gateway listener types * Improve variable name * Regenerate DeepCopy() code * Fix linting issue * fix protobuf import * Fix more merge conflict errors * Fix synthesize test * Run deep copy * Add URLRewrite to proto * Update agent/consul/discoverychain/gateway_tcproute.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Remove APIGatewayConfigEntry that was extra * Error out if route kind is unknown * Fix formatting errors in proto --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
Andrew Stucki	f4210d47dd	Add basic smoke test to make sure an APIGateway runs (#16217 )	2 years ago
Andrew Stucki	0891b4554d	Clean-up Gateway Controller Binding Logic (#16214 ) * Fix detecting when a route doesn't bind to a gateway because it's already bound * Clean up status setting code * rework binding a bit * More cleanup * Flatten all files * Fix up docstrings	2 years ago
skpratt	6f0b226b0d	ACL error improvements: incomplete bootstrapping and non-existent token (#16105 ) * add bootstrapping detail for acl errors * error detail improvements * update acl bootstrapping test coverage * update namespace errors * update test coverage * add changelog * update message for unbootstrapped error * consolidate error message code and update changelog * logout message change	2 years ago
Nathan Coleman	72a73661c9	Implement APIGateway proxycfg snapshot (#16194 ) * Stub proxycfg handler for API gateway * Add Service Kind constants/handling for API Gateway * Begin stubbing for SDS * Add new Secret type to xDS order of operations * Continue stubbing of SDS * Iterate on proxycfg handler for API gateway * Handle BoundAPIGateway config entry subscription in proxycfg-glue * Add API gateway to config snapshot validation * Add API gateway to config snapshot clone, leaf, etc. * Subscribe to bound route + cert config entries on bound-api-gateway * Track routes + certs on API gateway config snapshot * Generate DeepCopy() for types used in watch.Map * Watch all active references on api-gateway, unwatch inactive * Track loading of initial bound-api-gateway config entry * Use proper proto package for SDS mapping * Use ResourceReference instead of ServiceName, collect resources * Fix typo, add + remove TODOs * Watch discovery chains for TCPRoute * Add TODO for updating gateway services for api-gateway * make proto * Regenerate deep-copy for proxycfg * Set datacenter on upstream ID from query source * Watch discovery chains for http-route service backends * Add ServiceName getter to HTTP+TCP Service structs * Clean up unwatched discovery chains on API Gateway * Implement watch for ingress leaf certificate * Collect upstreams on http-route + tcp-route updates * Remove unused GatewayServices update handler * Remove unnecessary gateway services logic for API Gateway * Remove outdate TODO * Use .ToIngress where appropriate, including TODO for cleaning up * Cancel before returning error * Remove GatewayServices subscription * Add godoc for handlerAPIGateway functions * Update terminology from Connect => Consul Service Mesh Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690 * Add missing TODO * Remove duplicate switch case * Rerun deep-copy generator * Use correct property on config snapshot * Remove unnecessary leaf cert watch * Clean up based on code review feedback * Note handler properties that are initialized but set elsewhere * Add TODO for moving helper func into structs pkg * Update generated DeepCopy code * gofmt * Generate DeepCopy() for API gateway listener types * Improve variable name * Regenerate DeepCopy() code * Fix linting issue * Temporarily remove the secret type from resource generation	2 years ago
Nitya Dhanushkodi	1f25289048	troubleshoot: output messages for the troubleshoot proxy command (#16208 )	2 years ago
Kyle Havlovitz	898e59b13c	Add the `operator usage instances` command and api endpoint (#16205 ) This endpoint shows total services, connect service instances and billable service instances in the local datacenter or globally. Billable instances = total service instances - connect services - consul server instances.	2 years ago
Andrew Stucki	df03b45bbc	Add additional controller implementations (#16188 ) * Add additional controller implementations * remove additional interface * Fix comparison checks and mark unused contexts * Switch to time.Now().UTC() * Add a pointer helper for shadowing loop variables * Extract anonymous functions for readability * clean up logging * Add Type to the Condition proto * Update some comments and add additional space for readability * Address PR feedback * Fix up dirty checks and change to pointer receiver	2 years ago
Paul Banks	5397e9ee7f	Adding experimental support for a more efficient LogStore implementation (#16176 ) * Adding experimental support for a more efficient LogStore implementation * Adding changelog entry * Fix go mod tidy issues	2 years ago
cskh	e91bc9c058	feat: envoy extension - http local rate limit (#16196 ) - http local rate limit - Apply rate limit only to local_app - unit test and integ test	2 years ago
John Eikenberry	ed7367b6f4	remove redundant vault api retry logic (#16143 ) remove redundant vault api retry logic We upgraded Vault API module version to a version that has built-in retry logic. So this code is no longer necessary. Also add mention of re-configuring the provider in comments.	2 years ago
skpratt	1e7e52e3ef	revert method name change in xds server protocol for version compatibility (#16195 )	2 years ago
skpratt	9199e99e21	Update token language to distinguish Accessor and Secret ID usage (#16044 ) * remove legacy tokens * remove lingering legacy token references from docs * update language and naming for token secrets and accessor IDs * updates all tokenID references to clarify accessorID * remove token type references and lookup tokens by accessorID index * remove unnecessary constants * replace additional tokenID param names * Add warning info for deprecated -id parameter Co-authored-by: Paul Glass <pglass@hashicorp.com> * Update field comment Co-authored-by: Paul Glass <pglass@hashicorp.com> --------- Co-authored-by: Paul Glass <pglass@hashicorp.com>	2 years ago
wangxinyi7	906ebb97f6	change log level (#16128 )	2 years ago
Dhia Ayachi	c680a35b36	Net 2229/rpc reduce max retries 2 (#16165 ) * feat: calculate retry wait time with exponential back-off * test: add test for getWaitTime method * feat: enforce random jitter between min value from previous iteration and current * extract randomStagger to simplify tests and use Milliseconds to avoid float math. * rename variables * add test and rename comment --------- Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com>	2 years ago
Nitya Dhanushkodi	b8b37c2357	refactor: remove troubleshoot module dependency on consul top level module (#16162 ) Ensure nothing in the troubleshoot go module depends on consul's top level module. This is so we can import troubleshoot into consul-k8s and not import all of consul. * turns troubleshoot into a go module [authored by @curtbushko] * gets the envoy protos into the troubleshoot module [authored by @curtbushko] * adds a new go module `envoyextensions` which has xdscommon and extensioncommon folders that both the xds package and the troubleshoot package can import * adds testing and linting for the new go modules * moves the unit tests in `troubleshoot/validateupstream` that depend on proxycfg/xds into the xds package, with a comment describing why those tests cannot be in the troubleshoot package * fixes all the imports everywhere as a result of these changes Co-authored-by: Curt Bushko <cbushko@gmail.com>	2 years ago
Poonam Jadhav	24c431270c	feat: client RPC is retries on ErrRetryElsewhere error and forwardRequestToLeader method retries ErrRetryLater error (#16099 )	2 years ago
skpratt	a010902978	Remove legacy acl policies (#15922 ) * remove legacy tokens * remove legacy acl policies * flatten test policies to _prefix address oss feedback re: phrasing and tests	2 years ago
John Eikenberry	5c836f2aa9	fix goroutine leak in renew testing (#16142 ) fix goroutine leak in renew testing Test overwrote the stopWatcher() function variable for the test without keeping and calling the original value. The original value is the function that stops the goroutine... so it needs to be called.	2 years ago
sarahalsmiller	143b2bc1f0	API Gateway Controller Logic (#16058 ) * Add initial API gateway controller logic --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com> Co-authored-by: Thomas Eckert <teckert@hashicorp.com>	2 years ago
Derek Menteer	2f149d60cc	[OSS] Add Peer field to service-defaults upstream overrides (#15956 ) * Add Peer field to service-defaults upstream overrides. * add api changes, compat mode for service default overrides * Fixes based on testing --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>	2 years ago
Paul Glass	a884d0d7c7	Use agent token for service/check deregistration during anti-entropy (#16097 ) Use only the agent token for deregistration during anti-entropy The previous behavior had the agent attempt to use the "service" token (i.e. from the `token` field in a service definition file), and if that was not set then it would use the agent token. The previous behavior was problematic because, if the service token had been deleted, the deregistration request would fail. The agent would retry the deregistration during each anti-entropy sync, and the situation would never resolve. The new behavior is to only/always use the agent token for service and check deregistration during anti-entropy. This approach is: * Simpler: No fallback logic to try different tokens * Faster (slightly): No time spent attempting the service token * Correct: The agent token is able to deregister services on that agent's node, because: * node:write permissions allow deregistration of services/checks on that node. * The agent token must have node:write permission, or else the agent is not be able to (de)register itself into the catalog Co-authored-by: Vesa Hagström <weeezes@gmail.com>	2 years ago
Dan Upton	e40b731a52	rate: add prometheus definitions, docs, and clearer names (#15945 )	2 years ago
Nitya Dhanushkodi	8d4c3aa42c	refactor: move service to service validation to troubleshoot package (#16132 ) This is to reduce the dependency on xds from within the troubleshoot package.	2 years ago
Derek Menteer	06338c8ee7	Add unit test and update golden files. (#16115 )	2 years ago
Andrew Stucki	1fbfb5905b	APIGateway HTTPRoute scaffolding (#15859 ) * Stub Config Entries for Consul Native API Gateway (#15644) * Add empty InlineCertificate struct and protobuf * apigateway stubs * new files * Stub HTTPRoute in api pkg * checkpoint * Stub HTTPRoute in structs pkg * Simplify api.APIGatewayConfigEntry to be consistent w/ other entries * Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry * Add TCPRoute to MakeConfigEntry, return unique Kind * proto generated files * Stub BoundAPIGatewayConfigEntry in agent Since this type is only written by a controller and read by xDS, it doesn't need to be defined in the `api` pkg * Add RaftIndex to APIGatewayConfigEntry stub * Add new config entry kinds to validation allow-list * Add RaftIndex to other added config entry stubs * fix panic * Update usage metrics assertions to include new cfg entries * Regenerate proto w/ Go 1.19 * Run buf formatter on config_entry.proto * Add Meta and acl.EnterpriseMeta to all new ConfigEntry types * Remove optional interface method Warnings() for now Will restore later if we wind up needing it * Remove unnecessary Services field from added config entry types * Implement GetMeta(), GetEnterpriseMeta() for added config entry types * Add meta field to proto, name consistently w/ existing config entries * Format config_entry.proto * Add initial implementation of CanRead + CanWrite for new config entry types * Add unit tests for decoding of new config entry types * Add unit tests for parsing of new config entry types * Add unit tests for API Gateway config entry ACLs * Return typed PermissionDeniedError on BoundAPIGateway CanWrite * Add unit tests for added config entry ACLs * Add BoundAPIGateway type to AllConfigEntryKinds * Return proper kind from BoundAPIGateway * Add docstrings for new config entry types * Add missing config entry kinds to proto def * Update usagemetrics_oss_test.go * Use utility func for returning PermissionDeniedError * Add BoundAPIGateway to proto def Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Add APIGateway validation * Fix comment * Add additional validations * Add cert ref validation * Add protobuf definitions * Tabs to spaces * Fix up field types * Add API structs * Move struct fields around a bit * EventPublisher subscriptions for Consul Native API Gateway (#15757) * Create new event topics in subscribe proto * Add tests for PBSubscribe func * Make configs singular, add all configs to PBToStreamSubscribeRequest * Add snapshot methods * Add config_entry_events tests * Add config entry kind to topic for new configs * Add unit tests for snapshot methods * Start adding integration test * Test using the new controller code * Update agent/consul/state/config_entry_events.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Check value of error Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Add controller stubs for API Gateway (#15837) * update initial stub implementation * move files, clean up mutex references * Remove embed, use idiomatic names for constructors * Remove stray file introduced in merge Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Initial server-side and proto defs * drop trailing whitespace * Add APIGateway validation (#15847) * Add APIGateway validation * Fix comment * Add additional validations * Add cert ref validation * Add protobuf definitions * Tabs to spaces * Fix up field types * Add API structs * Move struct fields around a bit * APIGateway InlineCertificate validation (#15856) * Add APIGateway validation * Add additional validations * Add protobuf definitions * Tabs to spaces * Add API structs * Move struct fields around a bit * Add validation for InlineCertificate * Fix ACL test * APIGateway BoundAPIGateway validation (#15858) * Add APIGateway validation * Fix comment * Add additional validations * Add cert ref validation * Add protobuf definitions * Tabs to spaces * Fix up field types * Add API structs * Move struct fields around a bit * Add validation for BoundAPIGateway * drop trailing whitespace * APIGateway TCPRoute validation (#15855) * Add APIGateway validation * Fix comment * Add additional validations * Add cert ref validation * Add protobuf definitions * Tabs to spaces * Fix up field types * Add API structs * Move struct fields around a bit * Add TCPRoute normalization and validation * Address PR feedback * Add forgotten Status * Add some more field docs in api package * Fix test * Fix bad merge * Remove duplicate helpers * Fix up proto defs * Fix up stray changes * remove extra newline --------- Co-authored-by: Thomas Eckert <teckert@hashicorp.com> Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>	2 years ago
Derek Menteer	b19c5a94c7	Add Envoy extension metrics. (#16114 )	2 years ago
cskh	f6da81c9d0	improvement: prevent filter being added twice from any enovy extension (#16112 ) * improvement: prevent filter being added twice from any enovy extension * break if error != nil * update test	2 years ago
Poonam Jadhav	9db5b7d896	feat: apply retry policy to read only grpc endpoints (#16085 )	2 years ago
Derek Menteer	1b02749375	Add extension validation on config save and refactor extensions. (#16110 )	2 years ago
Nitya Dhanushkodi	8728a4496c	troubleshoot: service to service validation (#16096 ) * Add Tproxy support to Envoy Extensions (this is needed for service to service validation) * Add validation for Envoy configuration for an upstream service * Use both /config_dump and /cluster to validate Envoy configuration This is because of a bug in Envoy where the EndpointsConfigDump does not include a cluster_name, making it impossible to match an endpoint to verify it exists. This removes endpoints support for builtin extensions since only the validate plugin was using it, and it is no longer used. It also removes test cases for endpoint validation. Endpoints validation now only occurs in the top level test from config_dump and clusters json files. Co-authored-by: Eric <eric@haberkorn.co>	2 years ago
Andrew Stucki	da99514ac8	Add a server-only method for updating ConfigEntry Statuses (#16053 ) * Add a server-only method for updating ConfigEntry Statuses * Address PR feedback * Regen proto	2 years ago
skpratt	ad43846755	Remove legacy acl tokens (#15947 ) * remove legacy tokens * Update test comment Co-authored-by: Paul Glass <pglass@hashicorp.com> * fix imports * update docs for additional CLI changes * add test case for anonymous token * set deprecated api fields to json ignore and fix patch errors * update changelog to breaking-change * fix import * update api docs to remove legacy reference * fix docs nav data --------- Co-authored-by: Paul Glass <pglass@hashicorp.com>	2 years ago
Thomas Eckert	7814471159	Match route and listener protocols when binding (#16057 ) * Add GatewayMeta for matching routes to listeners based on protocols * Add GetGatewayMeta * Apply suggestions from code review Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Make GatewayMeta private * Bound -> BoundGateway * Document gatewayMeta more * Simplify conditional * Parallelize tests and simplify bind conditional * gofmt * 💧 getGatewayMeta --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>	2 years ago
Michael Wilkerson	a1498b015d	Mw/lambda envoy extension parse region (#4107 ) (#16069 ) * updated builtin extension to parse region directly from ARN - added a unit test - added some comments/light refactoring * updated golden files with proper ARNs - ARNs need to be right format now that they are being processed * updated tests and integration tests - removed 'region' from all EnvoyExtension arguments - added properly formatted ARN which includes the same region found in the removed "Region" field: 'us-east-1'	2 years ago
Andrew Stucki	3febdbff39	Add trigger for doing reconciliation based on watch sets (#16052 ) * Add trigger for doing reconciliation based on watch sets * update doc string * Fix my grammar fail	2 years ago
Poonam Jadhav	f4f62b5da6	feat: panic handler in rpc rate limit interceptor (#16022 ) * feat: handle panic in rpc rate limit interceptor * test: additional test cases to rpc rate limiting interceptor * refactor: remove unused listener	2 years ago
Nathan Coleman	e0f4f6c152	Run config entry controller routines on leader (#16054 )	2 years ago
Ronald	6167aef641	Warn when the token query param is used for auth (#16009 )	2 years ago
Thomas Eckert	20146f2916	Implement BindRoutesToGateways (#15950 ) * Stub out bind code * Move into a new package and flesh out binding * Fill in the actual binding logic * Bind to all listeners if not specified * Move bind code up to gateways package * Fix resource type check * Add UpsertRoute to listeners * Add RemoveRoute to listener * Implement binding as associated functions * Pass in gateways to BindRouteToGateways * Add a bunch of tests * Fix hopping from one listener on a gateway to another * Remove parents from HTTPRoute * Apply suggestions from code review * Fix merge conflict * Unify binding into a single variadic function 🙌 @nathancoleman * Remove vestigial error * Add TODO on protocol check	2 years ago
cskh	25396d81c9	Apply agent partition to load services and agent api (#16024 ) * Apply agent partition to load services and agent api changelog	2 years ago
Derek Menteer	5f5e6864ca	Fix proxy-defaults incorrectly merging config on upstreams. (#16021 )	2 years ago
John Murret	794277371f	Integration test for server rate limiting (#15960 ) * rate limit test * Have tests for the 3 modes * added assertions for logs and metrics * add comments to test sections * add check for rate limit exceeded text in log assertion section. * fix linting error * updating test to use KV get and put. move log assertion tolast. * Adding logging for blocking messages in enforcing mode. refactoring tests. * modified test description * formatting * Apply suggestions from code review Co-authored-by: Dan Upton <daniel@floppy.co> * Update test/integration/consul-container/test/ratelimit/ratelimit_test.go Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * expand log checking so that it ensures both logs are they when they are supposed to be and not there when they are not expected to be. * add retry on test * Warn once when rate limit exceed regardless of enforcing vs permissive. * Update test/integration/consul-container/test/ratelimit/ratelimit_test.go Co-authored-by: Dan Upton <daniel@floppy.co> Co-authored-by: Dan Upton <daniel@floppy.co> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago

... 2 3 4 5 6 ...

5183 Commits (698165858505eab8b0df029a9c2b551d891b9c92)