Commit Graph

27 Commits (48df56f7d2a517e9a08b479c0a9d54672f3e3a65)

Author SHA1 Message Date
Dhia Ayachi f2b26ac194
Hash based config entry replication (#19795)
* add a hash to config entries when normalizing

* add GetHash and implement comparing hashes

* only update if the Hash is different

* only update if the Hash is different and not 0

* fix proto to include the Hash

* fix proto gen

* buf format

* add SetHash and fix tests

* fix config load tests

* fix state test and config test

* recalculate hash when restoring config entries

* fix snapshot restore test

* add changelog

* fix missing normalize, fix proto indexes and add normalize test
2023-12-12 08:29:13 -05:00
John Murret f0cf8f2f40
NET-6294 - v1 Agentless proxycfg datasource errors after v2 changes (#19365) 2023-10-27 14:06:38 -06:00
Nathan Coleman e5d26a13cd
NET-5530 Support response header modifiers on http-route config entry (#18646)
* Add response header filters to http-route config entry definitions

* Map response header filters from config entry when constructing route destination

* Support response header modifiers at the service level as well

* Update protobuf definitions

* Update existing unit tests

* Add response filters to route consolidation logic

* Make existing unit tests more robust

* Add missing docstring

* Add changelog entry

* Add response filter modifiers to existing integration test

* Add more robust testing for response header modifiers in the discovery chain

* Add more robust testing for request header modifiers in the discovery chain

* Modify test to verify that service filter modifiers take precedence over rule filter modifiers
2023-09-08 14:04:56 -04:00
John Maguire 9876923e23
Add the plumbing for APIGW JWT work (#18609)
* Add the plumbing for APIGW JWT work

* Remove unneeded import

* Add deep equal function for HTTPMatch

* Added plumbing for status conditions

* Remove unneeded comment

* Fix comments

* Add calls in xds listener for apigateway to setup listener jwt auth
2023-08-31 12:23:59 -04:00
hashicorp-copywrite[bot] 5fb9df1640
[COMPLIANCE] License changes (#18443)
* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* add missing license headers

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
2023-08-11 09:12:13 -04:00
John Maguire df11e4e7b4
APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422)
* Updated httproute config entry for JWT Filters

* Added manual deepcopy method for httproute jwt filter

* Fix test

* Update JWTFilter to be in oss file

* Add changelog

* Add build tags for deepcopy oss file
2023-08-10 21:23:42 +00:00
sarahalsmiller e235c8be3c
NET-5115 Add retry + timeout filters for api-gateway (#18324)
* squash, implement retry/timeout in consul core

* update tests
2023-08-08 16:39:46 -05:00
John Maguire 92be8bd762
APIGW: Routes with duplicate parents should be invalid (#16926)
* ensure route parents are unique when creating an http route

* Ensure tcp route parents are unique

* Added unit tests
2023-04-10 13:20:32 -04:00
Ronald 94ec4eb2f4
copyright headers for agent folder (#16704)
* copyright headers for agent folder

* Ignore test data files

* fix proto files and remove headers in agent/uiserver folder

* ignore deep-copy files
2023-03-28 14:39:22 -04:00
John Maguire c833464daf
Update normalization of route refs (#16789)
* Use merge of enterprise meta's rather than new custom method

* Add merge logic for tcp routes

* Add changelog

* Normalize certificate refs on gateways

* Fix infinite call loop

* Explicitly call enterprise meta
2023-03-28 11:23:49 -04:00
John Maguire 351bdc3c0d
Fix struct tags for TCPService enterprise meta (#16781)
* Fix struct tags for TCPService enterprise meta

* Add changelog
2023-03-27 16:17:04 +00:00
John Maguire ff5887a99e
Update e2e tests for namespaces (#16627)
* Refactored "NewGatewayService" to handle namespaces, fixed
TestHTTPRouteFlattening test

* Fixed existing http_route tests for namespacing

* Squash aclEnterpriseMeta for ResourceRefs and HTTPServices, accept
namespace for creating connect services and regular services

* Use require instead of assert after creating namespaces in
http_route_tests

* Refactor NewConnectService and NewGatewayService functions to use cfg
objects to reduce number of method args

* Rename field on SidecarConfig in tests from `SidecarServiceName` to
`Name` to avoid stutter
2023-03-15 17:51:36 +00:00
Freddy 724b752ca7
Backport ENT-4704 (#16612) 2023-03-14 14:55:11 -06:00
Melisa Griffin 129eca8fdb
NET-2903 Normalize weight for http routes (#16512)
* NET-2903 Normalize weight for http routes

* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-03-03 16:39:59 -05:00
Andrew Stucki 0972697661
[API Gateway] Various fixes for Config Entry fields (#16347)
* [API Gateway] Various fixes for Config Entry fields

* simplify logic per PR review
2023-02-22 04:02:04 +00:00
Andrew Stucki 18e2ee77ca
[API Gateway] Fix targeting service splitters in HTTPRoutes (#16350)
* [API Gateway] Fix targeting service splitters in HTTPRoutes

* Fix test description
2023-02-22 03:48:26 +00:00
Andrew Stucki 15d2684ecc
Normalize all API Gateway references (#16316) 2023-02-17 21:37:34 +00:00
Andrew Stucki 58801cc8aa
Add stricter validation and some normalization code for API Gateway ConfigEntries (#16304)
* Add stricter validation and some normalization code for API Gateway ConfigEntries
2023-02-17 19:22:01 +00:00
Andrew Stucki e4a992c581
Fix hostname alignment checks for HTTPRoutes (#16300)
* Fix hostname alignment checks for HTTPRoutes
2023-02-17 18:18:11 +00:00
Thomas Eckert e81a0c2855
API Gateway to Ingress Gateway Snapshot Translation and Routes to Virtual Routers and Splitters (#16127)
* Stub proxycfg handler for API gateway

* Add Service Kind constants/handling for API Gateway

* Begin stubbing for SDS

* Add new Secret type to xDS order of operations

* Continue stubbing of SDS

* Iterate on proxycfg handler for API gateway

* Handle BoundAPIGateway config entry subscription in proxycfg-glue

* Add API gateway to config snapshot validation

* Add API gateway to config snapshot clone, leaf, etc.

* Subscribe to bound route + cert config entries on bound-api-gateway

* Track routes + certs on API gateway config snapshot

* Generate DeepCopy() for types used in watch.Map

* Watch all active references on api-gateway, unwatch inactive

* Track loading of initial bound-api-gateway config entry

* Use proper proto package for SDS mapping

* Use ResourceReference instead of ServiceName, collect resources

* Fix typo, add + remove TODOs

* Watch discovery chains for TCPRoute

* Add TODO for updating gateway services for api-gateway

* make proto

* Regenerate deep-copy for proxycfg

* Set datacenter on upstream ID from query source

* Watch discovery chains for http-route service backends

* Add ServiceName getter to HTTP+TCP Service structs

* Clean up unwatched discovery chains on API Gateway

* Implement watch for ingress leaf certificate

* Collect upstreams on http-route + tcp-route updates

* Remove unused GatewayServices update handler

* Remove unnecessary gateway services logic for API Gateway

* Remove outdate TODO

* Use .ToIngress where appropriate, including TODO for cleaning up

* Cancel before returning error

* Remove GatewayServices subscription

* Add godoc for handlerAPIGateway functions

* Update terminology from Connect => Consul Service Mesh

Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690

* Add missing TODO

* Remove duplicate switch case

* Rerun deep-copy generator

* Use correct property on config snapshot

* Remove unnecessary leaf cert watch

* Clean up based on code review feedback

* Note handler properties that are initialized but set elsewhere

* Add TODO for moving helper func into structs pkg

* Update generated DeepCopy code

* gofmt

* Begin stubbing for SDS

* Start adding tests

* Remove second BoundAPIGateway case in glue

* TO BE PICKED: fix formatting of str

* WIP

* Fix merge conflict

* Implement HTTP Route to Discovery Chain config entries

* Stub out function to create discovery chain

* Add discovery chain merging code (#16131)

* Test adding TCP and HTTP routes

* Add some tests for the synthesizer

* Run go mod tidy

* Pairing with N8

* Run deep copy

* Clean up GatewayChainSynthesizer

* Fix missing assignment of BoundAPIGateway topic

* Separate out synthesizeChains and toIngressTLS

* Fix build errors

* Ensure synthesizer skips non-matching routes by protocol

* Rebase on N8s work

* Generate DeepCopy() for API gateway listener types

* Improve variable name

* Regenerate DeepCopy() code

* Fix linting issue

* fix protobuf import

* Fix more merge conflict errors

* Fix synthesize test

* Run deep copy

* Add URLRewrite to proto

* Update agent/consul/discoverychain/gateway_tcproute.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Remove APIGatewayConfigEntry that was extra

* Error out if route kind is unknown

* Fix formatting errors in proto

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-02-09 17:58:55 +00:00
Nathan Coleman 72a73661c9
Implement APIGateway proxycfg snapshot (#16194)
* Stub proxycfg handler for API gateway

* Add Service Kind constants/handling for API Gateway

* Begin stubbing for SDS

* Add new Secret type to xDS order of operations

* Continue stubbing of SDS

* Iterate on proxycfg handler for API gateway

* Handle BoundAPIGateway config entry subscription in proxycfg-glue

* Add API gateway to config snapshot validation

* Add API gateway to config snapshot clone, leaf, etc.

* Subscribe to bound route + cert config entries on bound-api-gateway

* Track routes + certs on API gateway config snapshot

* Generate DeepCopy() for types used in watch.Map

* Watch all active references on api-gateway, unwatch inactive

* Track loading of initial bound-api-gateway config entry

* Use proper proto package for SDS mapping

* Use ResourceReference instead of ServiceName, collect resources

* Fix typo, add + remove TODOs

* Watch discovery chains for TCPRoute

* Add TODO for updating gateway services for api-gateway

* make proto

* Regenerate deep-copy for proxycfg

* Set datacenter on upstream ID from query source

* Watch discovery chains for http-route service backends

* Add ServiceName getter to HTTP+TCP Service structs

* Clean up unwatched discovery chains on API Gateway

* Implement watch for ingress leaf certificate

* Collect upstreams on http-route + tcp-route updates

* Remove unused GatewayServices update handler

* Remove unnecessary gateway services logic for API Gateway

* Remove outdate TODO

* Use .ToIngress where appropriate, including TODO for cleaning up

* Cancel before returning error

* Remove GatewayServices subscription

* Add godoc for handlerAPIGateway functions

* Update terminology from Connect => Consul Service Mesh

Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690

* Add missing TODO

* Remove duplicate switch case

* Rerun deep-copy generator

* Use correct property on config snapshot

* Remove unnecessary leaf cert watch

* Clean up based on code review feedback

* Note handler properties that are initialized but set elsewhere

* Add TODO for moving helper func into structs pkg

* Update generated DeepCopy code

* gofmt

* Generate DeepCopy() for API gateway listener types

* Improve variable name

* Regenerate DeepCopy() code

* Fix linting issue

* Temporarily remove the secret type from resource generation
2023-02-08 15:52:12 -06:00
Andrew Stucki df03b45bbc
Add additional controller implementations (#16188)
* Add additional controller implementations

* remove additional interface

* Fix comparison checks and mark unused contexts

* Switch to time.Now().UTC()

* Add a pointer helper for shadowing loop variables

* Extract anonymous functions for readability

* clean up logging

* Add Type to the Condition proto

* Update some comments and add additional space for readability

* Address PR feedback

* Fix up dirty checks and change to pointer receiver
2023-02-08 14:50:17 -05:00
Andrew Stucki 1fbfb5905b
APIGateway HTTPRoute scaffolding (#15859)
* Stub Config Entries for Consul Native API Gateway (#15644)

* Add empty InlineCertificate struct and protobuf

* apigateway stubs

* new files

* Stub HTTPRoute in api pkg

* checkpoint

* Stub HTTPRoute in structs pkg

* Simplify api.APIGatewayConfigEntry to be consistent w/ other entries

* Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry

* Add TCPRoute to MakeConfigEntry, return unique Kind

* proto generated files

* Stub BoundAPIGatewayConfigEntry in agent

Since this type is only written by a controller and read by xDS, it doesn't need to be defined in the `api` pkg

* Add RaftIndex to APIGatewayConfigEntry stub

* Add new config entry kinds to validation allow-list

* Add RaftIndex to other added config entry stubs

* fix panic

* Update usage metrics assertions to include new cfg entries

* Regenerate proto w/ Go 1.19

* Run buf formatter on config_entry.proto

* Add Meta and acl.EnterpriseMeta to all new ConfigEntry types

* Remove optional interface method Warnings() for now

Will restore later if we wind up needing it

* Remove unnecessary Services field from added config entry types

* Implement GetMeta(), GetEnterpriseMeta() for added config entry types

* Add meta field to proto, name consistently w/ existing config entries

* Format config_entry.proto

* Add initial implementation of CanRead + CanWrite for new config entry types

* Add unit tests for decoding of new config entry types

* Add unit tests for parsing of new config entry types

* Add unit tests for API Gateway config entry ACLs

* Return typed PermissionDeniedError on BoundAPIGateway CanWrite

* Add unit tests for added config entry ACLs

* Add BoundAPIGateway type to AllConfigEntryKinds

* Return proper kind from BoundAPIGateway

* Add docstrings for new config entry types

* Add missing config entry kinds to proto def

* Update usagemetrics_oss_test.go

* Use utility func for returning PermissionDeniedError

* Add BoundAPIGateway to proto def

Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Add APIGateway validation

* Fix comment

* Add additional validations

* Add cert ref validation

* Add protobuf definitions

* Tabs to spaces

* Fix up field types

* Add API structs

* Move struct fields around a bit

* EventPublisher subscriptions for Consul Native API Gateway (#15757)

* Create new event topics in subscribe proto
* Add tests for PBSubscribe func
* Make configs singular, add all configs to PBToStreamSubscribeRequest
* Add snapshot methods
* Add config_entry_events tests
* Add config entry kind to topic for new configs
* Add unit tests for snapshot methods
* Start adding integration test
* Test using the new controller code
* Update agent/consul/state/config_entry_events.go
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* Check value of error
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Add controller stubs for API Gateway (#15837)

* update initial stub implementation

* move files, clean up mutex references

* Remove embed, use idiomatic names for constructors

* Remove stray file introduced in merge

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Initial server-side and proto defs

* drop trailing whitespace

* Add APIGateway validation (#15847)

* Add APIGateway validation

* Fix comment

* Add additional validations

* Add cert ref validation

* Add protobuf definitions

* Tabs to spaces

* Fix up field types

* Add API structs

* Move struct fields around a bit

* APIGateway InlineCertificate validation (#15856)

* Add APIGateway validation

* Add additional validations

* Add protobuf definitions

* Tabs to spaces

* Add API structs

* Move struct fields around a bit

* Add validation for InlineCertificate

* Fix ACL test

* APIGateway BoundAPIGateway validation (#15858)

* Add APIGateway validation

* Fix comment

* Add additional validations

* Add cert ref validation

* Add protobuf definitions

* Tabs to spaces

* Fix up field types

* Add API structs

* Move struct fields around a bit

* Add validation for BoundAPIGateway

* drop trailing whitespace

* APIGateway TCPRoute validation (#15855)

* Add APIGateway validation

* Fix comment

* Add additional validations

* Add cert ref validation

* Add protobuf definitions

* Tabs to spaces

* Fix up field types

* Add API structs

* Move struct fields around a bit

* Add TCPRoute normalization and validation

* Address PR feedback

* Add forgotten Status

* Add some more field docs in api package

* Fix test

* Fix bad merge

* Remove duplicate helpers

* Fix up proto defs

* Fix up stray changes

* remove extra newline

---------

Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
2023-02-01 07:59:49 -05:00
Andrew Stucki da99514ac8
Add a server-only method for updating ConfigEntry Statuses (#16053)
* Add a server-only method for updating ConfigEntry Statuses

* Address PR feedback

* Regen proto
2023-01-27 14:34:11 -05:00
Thomas Eckert 7814471159
Match route and listener protocols when binding (#16057)
* Add GatewayMeta for matching routes to listeners based on protocols
* Add GetGatewayMeta
* Apply suggestions from code review
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* Make GatewayMeta private
* Bound -> BoundGateway
* Document gatewayMeta more
* Simplify conditional
* Parallelize tests and simplify bind conditional
* gofmt
* 💧 getGatewayMeta
---------
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-01-27 09:41:03 -05:00
Thomas Eckert 20146f2916
Implement BindRoutesToGateways (#15950)
* Stub out bind code
* Move into a new package and flesh out binding
* Fill in the actual binding logic
* Bind to all listeners if not specified
* Move bind code up to gateways package
* Fix resource type check
* Add UpsertRoute to listeners
* Add RemoveRoute to listener
* Implement binding as associated functions
* Pass in gateways to BindRouteToGateways
* Add a bunch of tests
* Fix hopping from one listener on a gateway to another
* Remove parents from HTTPRoute
* Apply suggestions from code review
* Fix merge conflict
* Unify binding into a single variadic function 🙌 @nathancoleman
* Remove vestigial error
* Add TODO on protocol check
2023-01-20 15:11:16 -05:00
Thomas Eckert 13da1a5285
Native API Gateway Config Entries (#15897)
* Stub Config Entries for Consul Native API Gateway (#15644)
* Add empty InlineCertificate struct and protobuf
* apigateway stubs
* Stub HTTPRoute in api pkg
* Stub HTTPRoute in structs pkg
* Simplify api.APIGatewayConfigEntry to be consistent w/ other entries
* Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry
* Add TCPRoute to MakeConfigEntry, return unique Kind
* Stub BoundAPIGatewayConfigEntry in agent
* Add RaftIndex to APIGatewayConfigEntry stub
* Add new config entry kinds to validation allow-list
* Add RaftIndex to other added config entry stubs
* Update usage metrics assertions to include new cfg entries
* Add Meta and acl.EnterpriseMeta to all new ConfigEntry types
* Remove unnecessary Services field from added config entry types
* Implement GetMeta(), GetEnterpriseMeta() for added config entry types
* Add meta field to proto, name consistently w/ existing config entries
* Format config_entry.proto
* Add initial implementation of CanRead + CanWrite for new config entry types
* Add unit tests for decoding of new config entry types
* Add unit tests for parsing of new config entry types
* Add unit tests for API Gateway config entry ACLs
* Return typed PermissionDeniedError on BoundAPIGateway CanWrite
* Add unit tests for added config entry ACLs
* Add BoundAPIGateway type to AllConfigEntryKinds
* Return proper kind from BoundAPIGateway
* Add docstrings for new config entry types
* Add missing config entry kinds to proto def
* Update usagemetrics_oss_test.go
* Use utility func for returning PermissionDeniedError
* EventPublisher subscriptions for Consul Native API Gateway (#15757)
* Create new event topics in subscribe proto
* Add tests for PBSubscribe func
* Make configs singular, add all configs to PBToStreamSubscribeRequest
* Add snapshot methods
* Add config_entry_events tests
* Add config entry kind to topic for new configs
* Add unit tests for snapshot methods
* Start adding integration test
* Test using the new controller code
* Update agent/consul/state/config_entry_events.go
* Check value of error
* Add controller stubs for API Gateway (#15837)
* update initial stub implementation
* move files, clean up mutex references
* Remove embed, use idiomatic names for constructors
* Remove stray file introduced in merge
* Add APIGateway validation (#15847)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* APIGateway InlineCertificate validation (#15856)
* Add APIGateway validation
* Add additional validations
* Add protobuf definitions
* Tabs to spaces
* Add API structs
* Move struct fields around a bit
* Add validation for InlineCertificate
* Fix ACL test
* APIGateway BoundAPIGateway validation (#15858)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* Add validation for BoundAPIGateway
* APIGateway TCPRoute validation (#15855)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Add TCPRoute normalization and validation
* Add forgotten Status
* Add some more field docs in api package
* Fix test
* Format imports
* Rename snapshot test variable names
* Add plumbing for Native API GW Subscriptions (#16003)

Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-01-18 22:14:34 +00:00