201 Commits (release/1.16.x)

Author	SHA1	Message	Date
hc-github-team-consul-core	6350270b7c	Backport of [NET-8367] security: upgrade google.golang.org/protobuf to 1.33.0 into release/1.16.x (#20804) ... * backport of commit a407616755 * backport of commit 455f67fa57 --------- Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>	9 months ago
hc-github-team-es-release-engineering	057c8db74b	[DO NOT MERGE UNTIL EOY] EOY license fixes 1.16.x (#19782)	11 months ago
hc-github-team-consul-core	dcb46458a8	Backport of Hash based config entry replication into release/1.16.x (#19915) ... add hash based config entry replication Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	12 months ago
lornasong	439f945b18	Manual Backport of [Cloud][CC-6925] Updates to pushing server state into release/1.16.x (#19818) ... * [Cloud][CC-6925] Updates to pushing server state (#19682) * Upgrade hcp-sdk-go to latest version v0.73 Changes: - go get github.com/hashicorp/hcp-sdk-go - go mod tidy * From upgrade: regenerate protobufs for upgrade from 1.30 to 1.31 Ran: `make proto` Slack: https://hashicorp.slack.com/archives/C0253EQ5B40/p1701105418579429 * From upgrade: fix mock interface implementation After upgrading, there is the following compile error: cannot use &mockHCPCfg{} (value of type *mockHCPCfg) as "github.com/hashicorp/hcp-sdk-go/config".HCPConfig value in return statement: *mockHCPCfg does not implement "github.com/hashicorp/hcp-sdk-go/config".HCPConfig (missing method Logout) Solution: update the mock to have the missing Logout method * From upgrade: Lint: remove usage of deprecated req.ServerState.TLS Due to upgrade, linting is erroring due to usage of a newly deprecated field 22:47:56 [consul]: make lint --> Running golangci-lint (.) agent/hcp/testing.go:157:24: SA1019: req.ServerState.TLS is deprecated: use server_tls.internal_rpc instead. (staticcheck) time.Until(time.Time(req.ServerState.TLS.CertExpiry)).Hours()/24, ^ * From upgrade: adjust oidc error message From the upgrade, this test started failing: === FAIL: internal/go-sso/oidcauth TestOIDC_ClaimsFromAuthCode/failed_code_exchange (re-run 2) (0.01s) oidc_test.go:393: unexpected error: Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code" Prior to the upgrade, the error returned was: ``` Provider login failed: Error exchanging oidc code: oauth2: cannot fetch token: 401 Unauthorized\nResponse: {\"error\":\"invalid_grant\",\"error_description\":\"unexpected auth code\"}\n ``` Now the error returned is as below and does not contain "cannot fetch token" ``` Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code" ``` * Update AgentPushServerState structs with new fields HCP-side changes for the new fields are in: https://github.com/hashicorp/cloud-global-network-manager-service/pull/1195/files * Minor refactor for hcpServerStatus to abstract tlsInfo into struct This will make it easier to set the same tls-info information to both - status.TLS (deprecated field) - status.ServerTLSMetadata (new field to use instead) * Update hcpServerStatus to parse out information for new fields Changes: - Improve error message and handling (encountered some issues and was confused) - Set new field TLSInfo.CertIssuer - Collect certificate authority metadata and set on TLSInfo.CertificateAuthorities - Set TLSInfo on both server.TLS and server.ServerTLSMetadata.InternalRPC * Update serverStatusToHCP to convert new fields to GNM rpc * Add changelog * Feedback: connect.ParseCert, caCerts * Feedback: refactor and unit test server status * Feedback: test to use expected struct * Feedback: certificate with intermediate * Feedback: catch no leaf, remove expectedErr * Feedback: update todos with jira ticket * Feedback: mock tlsConfigurator * Run make proto for files in 1.16 not in main * update licensing per 1.16 licensing	12 months ago
John Maguire	77035f7af2	[NET-5688] APIGateway UI Topology Fixes (#19657) Backport (#19763) ... * [NET-5688] APIGateway UI Topology Fixes (#19657) * Update catalog and ui endpoints to show APIGateway in gateway service topology view * Added initial implementation for service view * updated ui * Fix topology view for gateways * Adding tests for gw controller * remove unused args * Undo formatting changes * Fix call sites for upstream/downstream gw changes * Add config entry tests * Fix function calls again * Move from ServiceKey to ServiceName, cleanup from PR review * Add additional check for length of services in bound apigateway for IsSame comparison * fix formatting for proto * gofmt * Add DeepCopy for retrieved BoundAPIGateway * gofmt * gofmt * Rename function to be more consistent * Remove busl license	1 year ago
Nathan Coleman	32cf6ea961	Revert "Backport of NET-5530 Support response header modifiers on http-route config entry into release/1.16.x" (#18770) ... Revert "Backport of NET-5530 Support response header modifiers on http-route config entry into release/1.16.x (#18725)" This reverts commit b8a0e84983.	1 year ago
hc-github-team-consul-core	b8a0e84983	Backport of NET-5530 Support response header modifiers on http-route config entry into release/1.16.x (#18725) ... * NET-5530 Support response header modifiers on http-route config entry (#18646) * Add response header filters to http-route config entry definitions * Map response header filters from config entry when constructing route destination * Support response header modifiers at the service level as well * Update protobuf definitions * Update existing unit tests * Add response filters to route consolidation logic * Make existing unit tests more robust * Add missing docstring * Add changelog entry * Add response filter modifiers to existing integration test * Add more robust testing for response header modifiers in the discovery chain * Add more robust testing for request header modifiers in the discovery chain * Modify test to verify that service filter modifiers take precedence over rule filter modifiers * Generate deep-copy code --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>	1 year ago
Dan Bond	dce0ab6d0d	Manual Backport of Add TCP+TLS Healthchecks into release/1.16.x (#18678) ... Add TCP+TLS Healthchecks (#18381) * Begin adding TCPUseTLS * More TCP with TLS plumbing * Making forward progress * Keep on adding TCP+TLS support for healthchecks * Removed too many lines * Unit tests for TCP+TLS * Update tlsutil/config.go Co-authored-by: Samantha <hello@entropy.cat> * Working on the tcp+tls unit test * Updated the runtime integration tests * Progress * Revert this file back to HEAD * Remove debugging lines * Implement TLS enabled TCP socket server and make a successful TCP+TLS healthcheck on it * Update docs * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update agent/checks/check.go Co-authored-by: Samantha <hello@entropy.cat> * Address comments * Remove extraneous bracket * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update agent/agent_test.go Co-authored-by: Samantha <hello@entropy.cat> * Update website/content/docs/ecs/configuration-reference.mdx Co-authored-by: Samantha <hello@entropy.cat> * Update the mockTLSServer * Remove trailing newline * Address comments * Fix merge problem * Add changelog entry --------- Co-authored-by: Samantha <hello@entropy.cat> (cherry picked from commit 7ea986783d) Co-authored-by: Phil Porada <pgporada@users.noreply.github.com>	1 year ago
Semir Patel	7c72bba225	[BACKPORT] 1.16.x manual backport of OSS->CE branch (#18549)	1 year ago
Ronald	7408c14c99	Expose JWKS cluster config through JWTProviderConfigEntry (#17978) (#18002) ... * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA	1 year ago
hc-github-team-consul-core	9cb6679386	backport of commit 2735bbe60f (#17613) ... Co-authored-by: Eric <eric@haberkorn.co>	1 year ago
Matt Keeler	77f44fa878	Various bits of cleanup detected when using Go Workspaces (#17462) ... TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors. This commit: 1. Fixes envoy-library-references.sh to work again 2. Ensures we are pulling in go-control-plane@v0.11.0 everywhere (previously it was at that version in some modules and others were much older) 3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it. 4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field. 4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules 5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment. 6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible 7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner. 8. Updated linter rules to prevent usage of golang/protobuf 9. Updated a leader peering test to account for an extra colon in a grpc error message.	2 years ago
Dan Stough	d935c7b466	[OSS] gRPC Blocking Queries (#17426) ... * feat: initial grpc blocking queries * changelog and docs update	2 years ago
R.B. Boyer	e00280e7df	prototest: fix early return condition in AssertElementsMatch (#17416)	2 years ago
Ronald	113202d541	JWT Authentication with service intentions: xds package update (#17414) ... * JWT Authentication with service intentions: update xds package to translate config to envoy	2 years ago
Semir Patel	5eaeb7b8e5	Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks (#15979) ... * Add MaxEjectionPercent to config entry * Add BaseEjectionTime to config entry * Add MaxEjectionPercent and BaseEjectionTime to protobufs * Add MaxEjectionPercent and BaseEjectionTime to api * Fix integration test breakage * Verify MaxEjectionPercent and BaseEjectionTime in integration test upstream confings * Website docs for MaxEjectionPercent and BaseEjection time * Add `make docs` to browse docs at http://localhost:3000 * Changelog entry * so that is the difference between consul-docker and dev-docker * blah * update proto funcs * update proto --------- Co-authored-by: Maliz <maliheh.monshizadeh@hashicorp.com>	2 years ago
Eric Haberkorn	b1fae05983	Add sameness groups to service intentions. (#17064)	2 years ago
Ronald	e07c09dbfb	Fix generated proto files (#17063) ... * [COMPLIANCE] Add Copyright and License Headers * generate proto --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	2 years ago
hashicorp-copywrite[bot]	9f81fc01e9	[COMPLIANCE] Add Copyright and License Headers (#16854) ... Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>	2 years ago
Paul Glass	f4406e69b9	[NET-3091] Update service intentions to support jwt provider references (#17037) ... * [NET-3090] Add new JWT provider config entry * Add initial test cases * update validations for jwt-provider config entry fields * more validation * start improving tests * more tests * Normalize * Improve tests and move validate fns * usage test update * Add split between ent and oss for partitions * fix lint issues * Added retry backoff, fixed tests, removed unused defaults * take into account default partitions * use countTrue and add aliases * omit audiences if empty * fix failing tests * add omit-entry * Add JWT intentions * generate proto * fix deep copy issues * remove extra field * added some tests * more tests * add validation for creating existing jwt * fix nil issue * More tests, fix conflicts and improve memdb call * fix namespace * add aliases * consolidate errors, skip duplicate memdb calls * reworked iteration over config entries * logic improvements from review --------- Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>	2 years ago
Paul Glass	77ecff3209	Permissive mTLS (#17035) ... This implements permissive mTLS , which allows toggling services into "permissive" mTLS mode. Permissive mTLS mode allows incoming "non Consul-mTLS" traffic to be forward unmodified to the application. * Update service-defaults and proxy-defaults config entries with a MutualTLSMode field * Update the mesh config entry with an AllowEnablingPermissiveMutualTLS field and implement the necessary validation. AllowEnablingPermissiveMutualTLS must be true to allow changing to MutualTLSMode=permissive, but this does not require that all proxy-defaults and service-defaults are currently in strict mode. * Update xDS listener config to add a "permissive filter chain" when MutualTLSMode=permissive for a particular service. The permissive filter chain matches incoming traffic by the destination port. If the destination port matches the service port from the catalog, then no mTLS is required and the traffic sent is forwarded unmodified to the application.	2 years ago
Derek Menteer	87324c9ec8	Add PrioritizeByLocality to config entries. (#17007) ... This commit adds the PrioritizeByLocality field to both proxy-config and service-resolver config entries for locality-aware routing. The field is currently intended for enterprise only, and will be used to enable prioritization of service-mesh connections to services based on geographical region / zone.	2 years ago
Michael Wilkerson	0dd4ea2033	* added Sameness Group to proto files (#16998) ... - added Sameness Group to config entries - added Sameness Group to subscriptions * generated proto files * added Sameness Group events to the state store - added test cases * Refactored health RPC Client - moved code that is common to rpcclient under rpcclient common.go. This will help set us up to support future RPC clients * Refactored proxycfg glue views - Moved views to rpcclient config entry. This will allow us to reuse this code for a config entry client * added config entry RPC Client - Copied most of the testing code from rpcclient/health * hooked up new rpcclient in agent * fixed documentation and comments for clarity	2 years ago
Dan Upton	4fa2537b3b	Resource `Write` endpoint (#16786)	2 years ago
Dan Upton	671d5825ca	Raft storage backend (#16619)	2 years ago
Eric Haberkorn	0d1d2fc4c9	add order by locality failover to Consul enterprise (#16791)	2 years ago
Ronald	b64674623e	Copyright headers for missing files/folders (#16708) ... * copyright headers for agent folder	2 years ago
Dhia Ayachi	10df4d83aa	add ip rate limiter controller OSS parts (#16790)	2 years ago
Dan Upton	81df781e5f	Add storage backend interface and in-memory implementation (#16538) ... Introduces `storage.Backend`, which will serve as the interface between the Resource Service and the underlying storage system (Raft today, but in the future, who knows!). The primary design goal of this interface is to keep its surface area small, and push as much functionality as possible into the layers above, so that new implementations can be added with little effort, and easily proven to be correct. To that end, we also provide a suite of "conformance" tests that can be run against a backend implementation to check it behaves correctly. In this commit, we introduce an initial in-memory storage backend, which is suitable for tests and when running Consul in development mode. This backend is a thin wrapper around the `Store` type, which implements a resource database using go-memdb and our internal pub/sub system. `Store` will also be used to handle reads in our Raft backend, and in the future, used as a local cache for external storage systems.	2 years ago
Poonam Jadhav	9c64731a56	feat: add category annotation to RPC and gRPC methods (#16646)	2 years ago
Eric Haberkorn	eaa39f4ef5	add sameness group support to service resolver failover and redirects (#16664)	2 years ago
Eric Haberkorn	57e2493415	allow setting locality on services and nodes (#16581)	2 years ago
Eric Haberkorn	dbaf8bf49c	add agent locality and replicate it across peer streams (#16522)	2 years ago
Eric Haberkorn	5f81662066	Add support for failover policies (#16505)	2 years ago
Andrew Stucki	4b661d1e0c	Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable (#16495) ... * Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable * Regenerate golden files * Add RequestTimeout field * Add changelog entry	2 years ago
Andrew Stucki	0972697661	[API Gateway] Various fixes for Config Entry fields (#16347) ... * [API Gateway] Various fixes for Config Entry fields * simplify logic per PR review	2 years ago
Matt Keeler	085c0addc0	Protobuf Refactoring for Multi-Module Cleanliness (#16302) ... Protobuf Refactoring for Multi-Module Cleanliness This commit includes the following: Moves all packages that were within proto/ to proto/private Rewrites imports to account for the packages being moved Adds in buf.work.yaml to enable buf workspaces Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes) Why: In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage. There were some recent changes to have our own ratelimiting annotations. The two combined were not working when I was trying to use them together (attempting to rebase another branch) Buf workspaces should be the solution to the problem Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root. This resulted in proto file name conflicts in the Go global protobuf type registry. The solution to that was to add in a private/ directory into the path within the proto/ directory. That then required rewriting all the imports. Is this safe? AFAICT yes The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc) Other than imports, there were no changes to any generated code as a result of this.	2 years ago
Nathan Coleman	dd0ca4825c	Add inline-certificate as possible payload of config-entry wrapper (#16254) ... Co-authored-by: Andrew Stucki <3577250+andrewstucki@users.noreply.github.com>	2 years ago
Thomas Eckert	e81a0c2855	API Gateway to Ingress Gateway Snapshot Translation and Routes to Virtual Routers and Splitters (#16127) ... * Stub proxycfg handler for API gateway * Add Service Kind constants/handling for API Gateway * Begin stubbing for SDS * Add new Secret type to xDS order of operations * Continue stubbing of SDS * Iterate on proxycfg handler for API gateway * Handle BoundAPIGateway config entry subscription in proxycfg-glue * Add API gateway to config snapshot validation * Add API gateway to config snapshot clone, leaf, etc. * Subscribe to bound route + cert config entries on bound-api-gateway * Track routes + certs on API gateway config snapshot * Generate DeepCopy() for types used in watch.Map * Watch all active references on api-gateway, unwatch inactive * Track loading of initial bound-api-gateway config entry * Use proper proto package for SDS mapping * Use ResourceReference instead of ServiceName, collect resources * Fix typo, add + remove TODOs * Watch discovery chains for TCPRoute * Add TODO for updating gateway services for api-gateway * make proto * Regenerate deep-copy for proxycfg * Set datacenter on upstream ID from query source * Watch discovery chains for http-route service backends * Add ServiceName getter to HTTP+TCP Service structs * Clean up unwatched discovery chains on API Gateway * Implement watch for ingress leaf certificate * Collect upstreams on http-route + tcp-route updates * Remove unused GatewayServices update handler * Remove unnecessary gateway services logic for API Gateway * Remove outdate TODO * Use .ToIngress where appropriate, including TODO for cleaning up * Cancel before returning error * Remove GatewayServices subscription * Add godoc for handlerAPIGateway functions * Update terminology from Connect => Consul Service Mesh Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690 * Add missing TODO * Remove duplicate switch case * Rerun deep-copy generator * Use correct property on config snapshot * Remove unnecessary leaf cert watch * Clean up based on code review feedback * Note handler properties that are initialized but set elsewhere * Add TODO for moving helper func into structs pkg * Update generated DeepCopy code * gofmt * Begin stubbing for SDS * Start adding tests * Remove second BoundAPIGateway case in glue * TO BE PICKED: fix formatting of str * WIP * Fix merge conflict * Implement HTTP Route to Discovery Chain config entries * Stub out function to create discovery chain * Add discovery chain merging code (#16131) * Test adding TCP and HTTP routes * Add some tests for the synthesizer * Run go mod tidy * Pairing with N8 * Run deep copy * Clean up GatewayChainSynthesizer * Fix missing assignment of BoundAPIGateway topic * Separate out synthesizeChains and toIngressTLS * Fix build errors * Ensure synthesizer skips non-matching routes by protocol * Rebase on N8s work * Generate DeepCopy() for API gateway listener types * Improve variable name * Regenerate DeepCopy() code * Fix linting issue * fix protobuf import * Fix more merge conflict errors * Fix synthesize test * Run deep copy * Add URLRewrite to proto * Update agent/consul/discoverychain/gateway_tcproute.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Remove APIGatewayConfigEntry that was extra * Error out if route kind is unknown * Fix formatting errors in proto --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
Andrew Stucki	f4210d47dd	Add basic smoke test to make sure an APIGateway runs (#16217)	2 years ago
Andrew Stucki	df03b45bbc	Add additional controller implementations (#16188) ... * Add additional controller implementations * remove additional interface * Fix comparison checks and mark unused contexts * Switch to time.Now().UTC() * Add a pointer helper for shadowing loop variables * Extract anonymous functions for readability * clean up logging * Add Type to the Condition proto * Update some comments and add additional space for readability * Address PR feedback * Fix up dirty checks and change to pointer receiver	2 years ago
Derek Menteer	2f149d60cc	[OSS] Add Peer field to service-defaults upstream overrides (#15956) ... * Add Peer field to service-defaults upstream overrides. * add api changes, compat mode for service default overrides * Fixes based on testing --------- Co-authored-by: DanStough <dan.stough@hashicorp.com>	2 years ago
Michael Wilkerson	6e8716cfe7	Ent merge move envoy extension proto (#16126) ... * Mw/lambda envoy extension parse region (#4107) * updated builtin extension to parse region directly from ARN - added a unit test - added some comments/light refactoring * updated golden files with proper ARNs - ARNs need to be right format now that they are being processed * updated tests and integration tests - removed 'region' from all EnvoyExtension arguments - added properly formatted ARN which includes the same region found in the removed "Region" field: 'us-east-1' * regenerated proto files	2 years ago
Andrew Stucki	1fbfb5905b	APIGateway HTTPRoute scaffolding (#15859) ... * Stub Config Entries for Consul Native API Gateway (#15644) * Add empty InlineCertificate struct and protobuf * apigateway stubs * new files * Stub HTTPRoute in api pkg * checkpoint * Stub HTTPRoute in structs pkg * Simplify api.APIGatewayConfigEntry to be consistent w/ other entries * Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry * Add TCPRoute to MakeConfigEntry, return unique Kind * proto generated files * Stub BoundAPIGatewayConfigEntry in agent Since this type is only written by a controller and read by xDS, it doesn't need to be defined in the `api` pkg * Add RaftIndex to APIGatewayConfigEntry stub * Add new config entry kinds to validation allow-list * Add RaftIndex to other added config entry stubs * fix panic * Update usage metrics assertions to include new cfg entries * Regenerate proto w/ Go 1.19 * Run buf formatter on config_entry.proto * Add Meta and acl.EnterpriseMeta to all new ConfigEntry types * Remove optional interface method Warnings() for now Will restore later if we wind up needing it * Remove unnecessary Services field from added config entry types * Implement GetMeta(), GetEnterpriseMeta() for added config entry types * Add meta field to proto, name consistently w/ existing config entries * Format config_entry.proto * Add initial implementation of CanRead + CanWrite for new config entry types * Add unit tests for decoding of new config entry types * Add unit tests for parsing of new config entry types * Add unit tests for API Gateway config entry ACLs * Return typed PermissionDeniedError on BoundAPIGateway CanWrite * Add unit tests for added config entry ACLs * Add BoundAPIGateway type to AllConfigEntryKinds * Return proper kind from BoundAPIGateway * Add docstrings for new config entry types * Add missing config entry kinds to proto def * Update usagemetrics_oss_test.go * Use utility func for returning PermissionDeniedError * Add BoundAPIGateway to proto def Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Add APIGateway validation * Fix comment * Add additional validations * Add cert ref validation * Add protobuf definitions * Tabs to spaces * Fix up field types * Add API structs * Move struct fields around a bit * EventPublisher subscriptions for Consul Native API Gateway (#15757) * Create new event topics in subscribe proto * Add tests for PBSubscribe func * Make configs singular, add all configs to PBToStreamSubscribeRequest * Add snapshot methods * Add config_entry_events tests * Add config entry kind to topic for new configs * Add unit tests for snapshot methods * Start adding integration test * Test using the new controller code * Update agent/consul/state/config_entry_events.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Check value of error Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Add controller stubs for API Gateway (#15837) * update initial stub implementation * move files, clean up mutex references * Remove embed, use idiomatic names for constructors * Remove stray file introduced in merge Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Initial server-side and proto defs * drop trailing whitespace * Add APIGateway validation (#15847) * Add APIGateway validation * Fix comment * Add additional validations * Add cert ref validation * Add protobuf definitions * Tabs to spaces * Fix up field types * Add API structs * Move struct fields around a bit * APIGateway InlineCertificate validation (#15856) * Add APIGateway validation * Add additional validations * Add protobuf definitions * Tabs to spaces * Add API structs * Move struct fields around a bit * Add validation for InlineCertificate * Fix ACL test * APIGateway BoundAPIGateway validation (#15858) * Add APIGateway validation * Fix comment * Add additional validations * Add cert ref validation * Add protobuf definitions * Tabs to spaces * Fix up field types * Add API structs * Move struct fields around a bit * Add validation for BoundAPIGateway * drop trailing whitespace * APIGateway TCPRoute validation (#15855) * Add APIGateway validation * Fix comment * Add additional validations * Add cert ref validation * Add protobuf definitions * Tabs to spaces * Fix up field types * Add API structs * Move struct fields around a bit * Add TCPRoute normalization and validation * Address PR feedback * Add forgotten Status * Add some more field docs in api package * Fix test * Fix bad merge * Remove duplicate helpers * Fix up proto defs * Fix up stray changes * remove extra newline --------- Co-authored-by: Thomas Eckert <teckert@hashicorp.com> Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>	2 years ago
Derek Menteer	1b02749375	Add extension validation on config save and refactor extensions. (#16110)	2 years ago
Andrew Stucki	da99514ac8	Add a server-only method for updating ConfigEntry Statuses (#16053) ... * Add a server-only method for updating ConfigEntry Statuses * Address PR feedback * Regen proto	2 years ago
Thomas Eckert	13da1a5285	Native API Gateway Config Entries (#15897) ... * Stub Config Entries for Consul Native API Gateway (#15644) * Add empty InlineCertificate struct and protobuf * apigateway stubs * Stub HTTPRoute in api pkg * Stub HTTPRoute in structs pkg * Simplify api.APIGatewayConfigEntry to be consistent w/ other entries * Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry * Add TCPRoute to MakeConfigEntry, return unique Kind * Stub BoundAPIGatewayConfigEntry in agent * Add RaftIndex to APIGatewayConfigEntry stub * Add new config entry kinds to validation allow-list * Add RaftIndex to other added config entry stubs * Update usage metrics assertions to include new cfg entries * Add Meta and acl.EnterpriseMeta to all new ConfigEntry types * Remove unnecessary Services field from added config entry types * Implement GetMeta(), GetEnterpriseMeta() for added config entry types * Add meta field to proto, name consistently w/ existing config entries * Format config_entry.proto * Add initial implementation of CanRead + CanWrite for new config entry types * Add unit tests for decoding of new config entry types * Add unit tests for parsing of new config entry types * Add unit tests for API Gateway config entry ACLs * Return typed PermissionDeniedError on BoundAPIGateway CanWrite * Add unit tests for added config entry ACLs * Add BoundAPIGateway type to AllConfigEntryKinds * Return proper kind from BoundAPIGateway * Add docstrings for new config entry types * Add missing config entry kinds to proto def * Update usagemetrics_oss_test.go * Use utility func for returning PermissionDeniedError * EventPublisher subscriptions for Consul Native API Gateway (#15757) * Create new event topics in subscribe proto * Add tests for PBSubscribe func * Make configs singular, add all configs to PBToStreamSubscribeRequest * Add snapshot methods * Add config_entry_events tests * Add config entry kind to topic for new configs * Add unit tests for snapshot methods * Start adding integration test * Test using the new controller code * Update agent/consul/state/config_entry_events.go * Check value of error * Add controller stubs for API Gateway (#15837) * update initial stub implementation * move files, clean up mutex references * Remove embed, use idiomatic names for constructors * Remove stray file introduced in merge * Add APIGateway validation (#15847) * Add APIGateway validation * Add additional validations * Add cert ref validation * Add protobuf definitions * Fix up field types * Add API structs * Move struct fields around a bit * APIGateway InlineCertificate validation (#15856) * Add APIGateway validation * Add additional validations * Add protobuf definitions * Tabs to spaces * Add API structs * Move struct fields around a bit * Add validation for InlineCertificate * Fix ACL test * APIGateway BoundAPIGateway validation (#15858) * Add APIGateway validation * Add additional validations * Add cert ref validation * Add protobuf definitions * Fix up field types * Add API structs * Move struct fields around a bit * Add validation for BoundAPIGateway * APIGateway TCPRoute validation (#15855) * Add APIGateway validation * Add additional validations * Add cert ref validation * Add protobuf definitions * Fix up field types * Add API structs * Add TCPRoute normalization and validation * Add forgotten Status * Add some more field docs in api package * Fix test * Format imports * Rename snapshot test variable names * Add plumbing for Native API GW Subscriptions (#16003) Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
Matt Keeler	5afd4657ec	Protobuf Modernization (#15949) ... * Protobuf Modernization Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf Marshallers (protobuf and json) needed some changes to account for different APIs. Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package. This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces. * Fix go-mod-tidy make target to work on all modules	2 years ago
Matt Keeler	8b4eb0c8c1	Fix usage of protoc-gen-go-binary (#15920) ... * Fix usage of protoc-gen-go-binary * Pin protoc-gen-go-binary version	2 years ago
Dan Upton	d4c435856b	grpc: `protoc` plugin for generating gRPC rate limit specifications (#15564) ... Adds automation for generating the map of `gRPC Method Name → Rate Limit Type` used by the middleware introduced in #15550, and will ensure we don't forget to add new endpoints. Engineers must annotate their RPCs in the proto file like so: ``` rpc Foo(FooRequest) returns (FooResponse) { option (consul.internal.ratelimit.spec) = { operation_type: READ, }; } ``` When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will be installed that writes rate-limit specs as a JSON array to a file called `.ratelimit.tmp` (one per protobuf package/directory). After running Buf, `make proto` will execute a post-process script that will ingest all of the `.ratelimit.tmp` files and generate a Go file containing the mappings in the `agent/grpc-middleware` package. In the enterprise repository, it will write an additional file with the enterprise-only endpoints. If an engineer forgets to add the annotation to a new RPC, the plugin will return an error like so: ``` RPC Foo is missing rate-limit specification, fix it with: import "proto-public/annotations/ratelimit/ratelimit.proto"; service Bar { rpc Foo(...) returns (...) { option (hashicorp.consul.internal.ratelimit.spec) = { operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT, }; } } ``` In the future, this annotation can be extended to support rate-limit category (e.g. KV vs Catalog) and to determine the retry policy.	2 years ago