Commit Graph

334 Commits (ee51fc87b4bd14b351f79df518304fd83237858a)

Author SHA1 Message Date
sarahalsmiller b147323fb0
xds: Remove APIGateway ToIngress function (#17453)
* xds generation for routes api gateway

* Update gateway.go

* move buildHttpRoute into xds package

* Update agent/consul/discoverychain/gateway.go

* remove unneeded function

* convert http route code to only run for http protocol to future proof code path

* Update agent/consul/discoverychain/gateway.go

Co-authored-by: Mike Morris <mikemorris@users.noreply.github.com>

* fix tests, clean up http check logic

* clean up todo

* Fix casing in docstring

* Fix import block, adjust docstrings

* Rename func

* Consolidate docstring onto single line

* Remove ToIngress() conversion for APIGW, which generates its own xDS now

* update name and comment

* use constant value

* use constant

* rename readyUpstreams to readyListeners to better communicate what that function is doing

---------

Co-authored-by: Mike Morris <mikemorris@users.noreply.github.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-05-25 15:16:37 +00:00
Dan Stough d935c7b466
[OSS] gRPC Blocking Queries (#17426)
* feat: initial grpc blocking queries

* changelog and docs update
2023-05-23 17:29:10 -04:00
sarahalsmiller e2a81aa8bd
xds: generate listeners directly from API gateway snapshot (#17398)
* API Gateway XDS Primitives, endpoints and clusters (#17002)

* XDS primitive generation for endpoints and clusters

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* server_test

* deleted extra file

* add missing parents to test

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Routes for API Gateway (#17158)

* XDS primitive generation for endpoints and clusters

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* server_test

* deleted extra file

* add missing parents to test

* checkpoint

* delete extra file

* httproute flattening code

* linting issue

* so close on this, calling for tonight

* unit test passing

* add in header manip to virtual host

* upstream rebuild commented out

* Use consistent upstream name whether or not we're rebuilding

* Start working through route naming logic

* Fix typos in test descriptions

* Simplify route naming logic

* Simplify RebuildHTTPRouteUpstream

* Merge additional compiled discovery chains instead of overwriting

* Use correct chain for flattened route, clean up + add TODOs

* Remove empty conditional branch

* Restore previous variable declaration

Limit the scope of this PR

* Clean up, improve TODO

* add logging, clean up todos

* clean up function

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* checkpoint, skeleton, tests not passing

* checkpoint

* endpoints xds cluster configuration

* resources test fix

* fix reversion in resources_test

* checkpoint

* Update agent/proxycfg/api_gateway.go

Co-authored-by: John Maguire <john.maguire@hashicorp.com>

* unit tests passing

* gofmt

* add deterministic sorting to appease the unit test gods

* remove panic

* Find ready upstream matching listener instead of first in list

* Clean up, improve TODO

* Modify getReadyUpstreams to filter upstreams by listener (#17410)

Each listener would previously have all upstreams from any route that bound to the listener. This is problematic when a route bound to one listener also binds to other listeners and so includes upstreams for multiple listeners. The list for a given listener would then wind up including upstreams for other listeners.

* clean up todos, references to api gateway in listeners_ingress

* merge in Nathan's fix

* Update agent/consul/discoverychain/gateway.go

* cleanup current todos, remove snapshot manipulation from generation code

* Update agent/structs/config_entry_gateways.go

Co-authored-by: Thomas Eckert <teckert@hashicorp.com>

* Update agent/consul/discoverychain/gateway.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Update agent/consul/discoverychain/gateway.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Update agent/proxycfg/snapshot.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* clarified header comment for FlattenHTTPRoute, changed RebuildHTTPRouteUpstream to BuildHTTPRouteUpstream

* simplify cert logic

* Delete scratch

* revert route related changes in listener PR

* Update agent/consul/discoverychain/gateway.go

* Update agent/proxycfg/snapshot.go

* clean up uneeded extra lines in endpoints

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
2023-05-22 17:36:29 -04:00
Ronald 113202d541
JWT Authentication with service intentions: xds package update (#17414)
* JWT Authentication with service intentions: update xds package to translate config to envoy
2023-05-19 18:14:16 -04:00
sarahalsmiller 134aac7c26
xds: generate endpoints directly from API gateway snapshot (#17390)
* endpoints xds cluster configuration

* resources test fix

* fix reversion in resources_test

* Update agent/proxycfg/api_gateway.go

Co-authored-by: John Maguire <john.maguire@hashicorp.com>

* gofmt

* Modify getReadyUpstreams to filter upstreams by listener (#17410)

Each listener would previously have all upstreams from any route that bound to the listener. This is problematic when a route bound to one listener also binds to other listeners and so includes upstreams for multiple listeners. The list for a given listener would then wind up including upstreams for other listeners.

* Update agent/proxycfg/api_gateway.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Restore import blocking

* Skip to next route if route has no upstreams

* cleanup

* change set from bool to empty struct

---------

Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-05-19 18:50:59 +00:00
Kyle Havlovitz 2904d0a431
Pull virtual IPs for filter chains from discovery chains (#17375) 2023-05-17 11:18:39 -07:00
Connor 0789661ce5
Rename hcp-metrics-collector to consul-telemetry-collector (#17327)
* Rename hcp-metrics-collector to consul-telemetry-collector

* Fix docs

* Fix doc comment

---------

Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>
2023-05-16 14:36:05 -04:00
Freddy 7c3e9cd862
Hash namespace+proxy ID when creating socket path (#17204)
UNIX domain socket paths are limited to 104-108 characters, depending on
the OS. This limit was quite easy to exceed when testing the feature on
Kubernetes, due to how proxy IDs encode the Pod ID eg:
metrics-collector-59467bcb9b-fkkzl-hcp-metrics-collector-sidecar-proxy

To ensure we stay under that character limit this commit makes a
couple changes:
- Use a b64 encoded SHA1 hash of the namespace + proxy ID to create a
  short and deterministic socket file name.
- Add validation to proxy registrations and proxy-defaults to enforce a
  limit on the socket directory length.
2023-05-09 12:20:26 -06:00
Derek Menteer 4f6da20fe5
Fix multiple issues related to proxycfg health queries. (#17241)
Fix multiple issues related to proxycfg health queries.

1. The datacenter was not being provided to a proxycfg query, which resulted in
bypassing agentless query optimizations and using the normal API instead.

2. The health rpc endpoint would return a zero index when insufficient ACLs were
detected. This would result in the agent cache performing an infinite loop of
queries in rapid succession without backoff.
2023-05-09 12:37:58 -05:00
Semir Patel 5eaeb7b8e5
Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks (#15979)
* Add MaxEjectionPercent to config entry

* Add BaseEjectionTime to config entry

* Add MaxEjectionPercent and BaseEjectionTime to protobufs

* Add MaxEjectionPercent and BaseEjectionTime to api

* Fix integration test breakage

* Verify MaxEjectionPercent and BaseEjectionTime in integration test upstream confings

* Website docs for MaxEjectionPercent and BaseEjection time

* Add `make docs` to browse docs at http://localhost:3000

* Changelog entry

* so that is the difference between consul-docker and dev-docker

* blah

* update proto funcs

* update proto

---------

Co-authored-by: Maliz <maliheh.monshizadeh@hashicorp.com>
2023-04-26 15:59:48 -07:00
Eric Haberkorn b1fae05983
Add sameness groups to service intentions. (#17064) 2023-04-20 12:16:04 -04:00
Eric Haberkorn 44b39240a8
move enterprise test cases out of open source (#16985) 2023-04-13 09:07:06 -04:00
cskh a319953576
docs: add envoy to the proxycfg diagram (#16834)
* docs: add envoy to the proxycfg diagram
2023-04-04 09:42:42 -04:00
Eric Haberkorn a6d69adcf5
Add default resolvers to disco chains based on the default sameness group (#16837) 2023-03-31 14:35:56 -04:00
Eric Haberkorn 0d1d2fc4c9
add order by locality failover to Consul enterprise (#16791) 2023-03-30 10:08:38 -04:00
Ronald 94ec4eb2f4
copyright headers for agent folder (#16704)
* copyright headers for agent folder

* Ignore test data files

* fix proto files and remove headers in agent/uiserver folder

* ignore deep-copy files
2023-03-28 14:39:22 -04:00
Derek Menteer 2236975011
Change partition for peers in discovery chain targets (#16769)
This commit swaps the partition field to the local partition for
discovery chains targeting peers. Prior to this change, peer upstreams
would always use a value of default regardless of which partition they
exist in. This caused several issues in xds / proxycfg because of id
mismatches.

Some prior fixes were made to deal with one-off id mismatches that this
PR also cleans up, since they are no longer needed.
2023-03-24 15:40:19 -05:00
Eric Haberkorn 495ad4c7ef
add enterprise xds tests (#16738) 2023-03-22 14:56:18 -04:00
John Maguire 8dd1d73874
Remove unused are hosts set check (#16691)
* Remove unused are hosts set check

* Remove all traces of unused 'AreHostsSet' parameter

* Remove unused Hosts attribute

* Remove commented out use of snap.APIGateway.Hosts
2023-03-21 16:23:23 +00:00
Nitya Dhanushkodi b9bd2c3780
peering: peering partition failover fixes (#16673)
add local source partition for peered upstreams
2023-03-20 10:00:29 -07:00
John Maguire 1ef9f4dade
Fix route subscription when using namespaces (#16677)
* Fix route subscription when using namespaces

* Update changelog

* Fix changelog entry to reference that the bug was enterprise only
2023-03-20 12:42:30 -04:00
Ashvitha f95ffe0355
Allow HCP metrics collection for Envoy proxies
Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>

Add a new envoy flag: "envoy_hcp_metrics_bind_socket_dir", a directory
where a unix socket will be created with the name
`<namespace>_<proxy_id>.sock` to forward Envoy metrics.

If set, this will configure:
- In bootstrap configuration a local stats_sink and static cluster.
  These will forward metrics to a loopback listener sent over xDS.

- A dynamic listener listening at the socket path that the previously
  defined static cluster is sending metrics to.

- A dynamic cluster that will forward traffic received at this listener
  to the hcp-metrics-collector service.


Reasons for having a static cluster pointing at a dynamic listener:
- We want to secure the metrics stream using TLS, but the stats sink can
  only be defined in bootstrap config. With dynamic listeners/clusters
  we can use the proxy's leaf certificate issued by the Connect CA,
  which isn't available at bootstrap time.

- We want to intelligently route to the HCP collector. Configuring its
  addreess at bootstrap time limits our flexibility routing-wise. More
  on this below.

Reasons for defining the collector as an upstream in `proxycfg`:
- The HCP collector will be deployed as a mesh service.

- Certificate management is taken care of, as mentioned above.

- Service discovery and routing logic is automatically taken care of,
  meaning that no code changes are required in the xds package.

- Custom routing rules can be added for the collector using discovery
  chain config entries. Initially the collector is expected to be
  deployed to each admin partition, but in the future could be deployed
  centrally in the default partition. These config entries could even be
  managed by HCP itself.
2023-03-10 13:52:54 -07:00
R.B. Boyer 9a485cdb49
proxycfg: ensure that an irrecoverable error in proxycfg closes the xds session and triggers a replacement proxycfg watcher (#16497)
Receiving an "acl not found" error from an RPC in the agent cache and the
streaming/event components will cause any request loops to cease under the
assumption that they will never work again if the token was destroyed. This
prevents log spam (#14144, #9738).

Unfortunately due to things like:

- authz requests going to stale servers that may not have witnessed the token
  creation yet

- authz requests in a secondary datacenter happening before the tokens get
  replicated to that datacenter

- authz requests from a primary TO a secondary datacenter happening before the
  tokens get replicated to that datacenter

The caller will get an "acl not found" *before* the token exists, rather than
just after. The machinery added above in the linked PRs will kick in and
prevent the request loop from looping around again once the tokens actually
exist.

For `consul-dataplane` usages, where xDS is served by the Consul servers
rather than the clients ultimately this is not a problem because in that
scenario the `agent/proxycfg` machinery is on-demand and launched by a new xDS
stream needing data for a specific service in the catalog. If the watching
goroutines are terminated it ripples down and terminates the xDS stream, which
CDP will eventually re-establish and restart everything.

For Consul client usages, the `agent/proxycfg` machinery is ahead-of-time
launched at service registration time (called "local" in some of the proxycfg
machinery) so when the xDS stream comes in the data is already ready to go. If
the watching goroutines terminate it should terminate the xDS stream, but
there's no mechanism to re-spawn the watching goroutines. If the xDS stream
reconnects it will see no `ConfigSnapshot` and will not get one again until
the client agent is restarted, or the service is re-registered with something
changed in it.

This PR fixes a few things in the machinery:

- there was an inadvertent deadlock in fetching snapshot from the proxycfg
  machinery by xDS, such that when the watching goroutine terminated the
  snapshots would never be fetched. This caused some of the xDS machinery to
  get indefinitely paused and not finish the teardown properly.

- Every 30s we now attempt to re-insert all locally registered services into
  the proxycfg machinery.

- When services are re-inserted into the proxycfg machinery we special case
  "dead" ones such that we unilaterally replace them rather that doing that
  conditionally.
2023-03-03 14:27:53 -06:00
Andrew Stucki 5deffbd95b
Fix issue where terminating gateway service resolvers weren't properly cleaned up (#16498)
* Fix issue where terminating gateway service resolvers weren't properly cleaned up

* Add integration test for cleaning up resolvers

* Add changelog entry

* Use state test and drop integration test
2023-03-03 09:56:57 -05:00
Andrew Stucki 4b661d1e0c
Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable (#16495)
* Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable

* Regenerate golden files

* Add RequestTimeout field

* Add changelog entry
2023-03-03 09:37:12 -05:00
Matt Keeler 085c0addc0
Protobuf Refactoring for Multi-Module Cleanliness (#16302)
Protobuf Refactoring for Multi-Module Cleanliness

This commit includes the following:

Moves all packages that were within proto/ to proto/private
Rewrites imports to account for the packages being moved
Adds in buf.work.yaml to enable buf workspaces
Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml
Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes)

Why:

In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage.
There were some recent changes to have our own ratelimiting annotations.
The two combined were not working when I was trying to use them together (attempting to rebase another branch)
Buf workspaces should be the solution to the problem
Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root.
This resulted in proto file name conflicts in the Go global protobuf type registry.
The solution to that was to add in a private/ directory into the path within the proto/ directory.
That then required rewriting all the imports.

Is this safe?

AFAICT yes
The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc)
Other than imports, there were no changes to any generated code as a result of this.
2023-02-17 16:14:46 -05:00
Andrew Stucki e4a992c581
Fix hostname alignment checks for HTTPRoutes (#16300)
* Fix hostname alignment checks for HTTPRoutes
2023-02-17 18:18:11 +00:00
Andrew Stucki b3ddd4d24e
Inline API Gateway TLS cert code (#16295)
* Include secret type when building resources from config snapshot

* First pass at generating envoy secrets from api-gateway snapshot

* Update comments for xDS update order

* Add secret type + corresponding golden files to existing tests

* Initialize test helpers for testing api-gateway resource generation

* Generate golden files for new api-gateway xDS resource test

* Support ADS for TLS certificates on api-gateway

* Configure TLS on api-gateway listeners

* Inline TLS cert code

* update tests

* Add SNI support so we can have multiple certificates

* Remove commented out section from helper

* regen deep-copy

* Add tcp tls test

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-02-17 12:46:03 -05:00
Thomas Eckert 2460ac99c9
API Gateway Envoy Golden Listener Tests (#16221)
* Simple API Gateway e2e test for tcp routes

* Drop DNSSans since we don't front the Gateway with a leaf cert

* WIP listener tests for api-gateway

* Return early if no routes

* Add back in leaf cert to testing

* Fix merge conflicts

* Re-add kind to setup

* Fix iteration over listener upstreams

* New tcp listener test

* Add tests for API Gateway with TCP and HTTP routes

* Move zero-route check back

* Drop generateIngressDNSSANs

* Check for chains not routes

---------

Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-02-16 14:42:36 -05:00
Derek Menteer 30112288c8
Fix mesh gateways incorrectly matching peer locality. (#16257)
Fix mesh gateways incorrectly matching peer locality.

This fixes an issue where local mesh gateways use an
incorrect address when attempting to forward traffic to a
peered datacenter. Prior to this change it would use the
lan address instead of the wan if the locality matched. This
should never be done for peering, since we must route all
traffic through the remote mesh gateway.
2023-02-16 09:22:41 -06:00
Derek Menteer 6599a9be1d
Fix nil-pointer panics from proxycfg package. (#16277)
Prior to this PR, servers / agents would panic and crash if an ingress
or api gateway were configured to use a discovery chain that both:

1. Referenced a peered service
2. Had a mesh gateway mode of local

This could occur, because code for handling upstream watches was shared
between both connect-proxy and the gateways. As a short-term fix, this
PR ensures that the maps are always initialized for these gateway services.

This PR also wraps the proxycfg execution and service
registration calls with recover statements to ensure that future issues
like this do not put the server into an unrecoverable state.
2023-02-15 11:54:44 -06:00
Andrew Stucki 9bb0ecfc18
[API Gateway] Add integration test for HTTP routes (#16236)
* [API Gateway] Add integration test for conflicted TCP listeners

* [API Gateway] Update simple test to leverage intentions and multiple listeners

* Fix broken unit test

* [API Gateway] Add integration test for HTTP routes
2023-02-13 14:18:05 -05:00
Andrew Stucki 8ff2974dbe
[API Gateway] Update simple test to leverage intentions and multiple listeners (#16228)
* [API Gateway] Add integration test for conflicted TCP listeners

* [API Gateway] Update simple test to leverage intentions and multiple listeners

* Fix broken unit test

* PR suggestions
2023-02-10 21:13:44 +00:00
Andrew Stucki 4c848a554d
Fix missing references to enterprise metadata (#16237) 2023-02-10 20:47:16 +00:00
Andrew Stucki 318ba215ab
[API Gateway] Add integration test for conflicted TCP listeners (#16225) 2023-02-10 11:34:01 -06:00
Andrew Stucki 3b9c569561
Simple API Gateway e2e test for tcp routes (#16222)
* Simple API Gateway e2e test for tcp routes

* Drop DNSSans since we don't front the Gateway with a leaf cert
2023-02-09 16:20:12 -05:00
Thomas Eckert e81a0c2855
API Gateway to Ingress Gateway Snapshot Translation and Routes to Virtual Routers and Splitters (#16127)
* Stub proxycfg handler for API gateway

* Add Service Kind constants/handling for API Gateway

* Begin stubbing for SDS

* Add new Secret type to xDS order of operations

* Continue stubbing of SDS

* Iterate on proxycfg handler for API gateway

* Handle BoundAPIGateway config entry subscription in proxycfg-glue

* Add API gateway to config snapshot validation

* Add API gateway to config snapshot clone, leaf, etc.

* Subscribe to bound route + cert config entries on bound-api-gateway

* Track routes + certs on API gateway config snapshot

* Generate DeepCopy() for types used in watch.Map

* Watch all active references on api-gateway, unwatch inactive

* Track loading of initial bound-api-gateway config entry

* Use proper proto package for SDS mapping

* Use ResourceReference instead of ServiceName, collect resources

* Fix typo, add + remove TODOs

* Watch discovery chains for TCPRoute

* Add TODO for updating gateway services for api-gateway

* make proto

* Regenerate deep-copy for proxycfg

* Set datacenter on upstream ID from query source

* Watch discovery chains for http-route service backends

* Add ServiceName getter to HTTP+TCP Service structs

* Clean up unwatched discovery chains on API Gateway

* Implement watch for ingress leaf certificate

* Collect upstreams on http-route + tcp-route updates

* Remove unused GatewayServices update handler

* Remove unnecessary gateway services logic for API Gateway

* Remove outdate TODO

* Use .ToIngress where appropriate, including TODO for cleaning up

* Cancel before returning error

* Remove GatewayServices subscription

* Add godoc for handlerAPIGateway functions

* Update terminology from Connect => Consul Service Mesh

Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690

* Add missing TODO

* Remove duplicate switch case

* Rerun deep-copy generator

* Use correct property on config snapshot

* Remove unnecessary leaf cert watch

* Clean up based on code review feedback

* Note handler properties that are initialized but set elsewhere

* Add TODO for moving helper func into structs pkg

* Update generated DeepCopy code

* gofmt

* Begin stubbing for SDS

* Start adding tests

* Remove second BoundAPIGateway case in glue

* TO BE PICKED: fix formatting of str

* WIP

* Fix merge conflict

* Implement HTTP Route to Discovery Chain config entries

* Stub out function to create discovery chain

* Add discovery chain merging code (#16131)

* Test adding TCP and HTTP routes

* Add some tests for the synthesizer

* Run go mod tidy

* Pairing with N8

* Run deep copy

* Clean up GatewayChainSynthesizer

* Fix missing assignment of BoundAPIGateway topic

* Separate out synthesizeChains and toIngressTLS

* Fix build errors

* Ensure synthesizer skips non-matching routes by protocol

* Rebase on N8s work

* Generate DeepCopy() for API gateway listener types

* Improve variable name

* Regenerate DeepCopy() code

* Fix linting issue

* fix protobuf import

* Fix more merge conflict errors

* Fix synthesize test

* Run deep copy

* Add URLRewrite to proto

* Update agent/consul/discoverychain/gateway_tcproute.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Remove APIGatewayConfigEntry that was extra

* Error out if route kind is unknown

* Fix formatting errors in proto

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-02-09 17:58:55 +00:00
Nathan Coleman 72a73661c9
Implement APIGateway proxycfg snapshot (#16194)
* Stub proxycfg handler for API gateway

* Add Service Kind constants/handling for API Gateway

* Begin stubbing for SDS

* Add new Secret type to xDS order of operations

* Continue stubbing of SDS

* Iterate on proxycfg handler for API gateway

* Handle BoundAPIGateway config entry subscription in proxycfg-glue

* Add API gateway to config snapshot validation

* Add API gateway to config snapshot clone, leaf, etc.

* Subscribe to bound route + cert config entries on bound-api-gateway

* Track routes + certs on API gateway config snapshot

* Generate DeepCopy() for types used in watch.Map

* Watch all active references on api-gateway, unwatch inactive

* Track loading of initial bound-api-gateway config entry

* Use proper proto package for SDS mapping

* Use ResourceReference instead of ServiceName, collect resources

* Fix typo, add + remove TODOs

* Watch discovery chains for TCPRoute

* Add TODO for updating gateway services for api-gateway

* make proto

* Regenerate deep-copy for proxycfg

* Set datacenter on upstream ID from query source

* Watch discovery chains for http-route service backends

* Add ServiceName getter to HTTP+TCP Service structs

* Clean up unwatched discovery chains on API Gateway

* Implement watch for ingress leaf certificate

* Collect upstreams on http-route + tcp-route updates

* Remove unused GatewayServices update handler

* Remove unnecessary gateway services logic for API Gateway

* Remove outdate TODO

* Use .ToIngress where appropriate, including TODO for cleaning up

* Cancel before returning error

* Remove GatewayServices subscription

* Add godoc for handlerAPIGateway functions

* Update terminology from Connect => Consul Service Mesh

Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690

* Add missing TODO

* Remove duplicate switch case

* Rerun deep-copy generator

* Use correct property on config snapshot

* Remove unnecessary leaf cert watch

* Clean up based on code review feedback

* Note handler properties that are initialized but set elsewhere

* Add TODO for moving helper func into structs pkg

* Update generated DeepCopy code

* gofmt

* Generate DeepCopy() for API gateway listener types

* Improve variable name

* Regenerate DeepCopy() code

* Fix linting issue

* Temporarily remove the secret type from resource generation
2023-02-08 15:52:12 -06:00
Derek Menteer 1b02749375
Add extension validation on config save and refactor extensions. (#16110) 2023-01-30 15:35:26 -06:00
Nitya Dhanushkodi 8728a4496c
troubleshoot: service to service validation (#16096)
* Add Tproxy support to Envoy Extensions (this is needed for service to service validation)

* Add validation for Envoy configuration for an upstream service

* Use both /config_dump and /cluster to validate Envoy configuration
This is because of a bug in Envoy where the EndpointsConfigDump does not
include a cluster_name, making it impossible to match an endpoint to
verify it exists.

This removes endpoints support for builtin extensions since only the
validate plugin was using it, and it is no longer used. It also removes
test cases for endpoint validation. Endpoints validation now only occurs
in the top level test from config_dump and clusters json files.

Co-authored-by: Eric <eric@haberkorn.co>
2023-01-27 11:43:16 -08:00
Michael Wilkerson a1498b015d
Mw/lambda envoy extension parse region (#4107) (#16069)
* updated builtin extension to parse region directly from ARN
- added a unit test
- added some comments/light refactoring

* updated golden files with proper ARNs
- ARNs need to be right format now that they are being processed

* updated tests and integration tests
- removed 'region' from all EnvoyExtension arguments
- added properly formatted ARN which includes the same region found in the removed "Region" field: 'us-east-1'
2023-01-26 15:44:52 -08:00
Derek Menteer 2facf50923
Fix configuration merging for implicit tproxy upstreams. (#16000)
Fix configuration merging for implicit tproxy upstreams.

Change the merging logic so that the wildcard upstream has correct proxy-defaults
and service-defaults values combined into it. It did not previously merge all fields,
and the wildcard upstream did not exist unless service-defaults existed (it ignored
proxy-defaults, essentially).

Change the way we fetch upstream configuration in the xDS layer so that it falls back
to the wildcard when no matching upstream is found. This is what allows implicit peer
upstreams to have the correct "merged" config.

Change proxycfg to always watch local mesh gateway endpoints whenever a peer upstream
is found. This simplifies the logic so that we do not have to inspect the "merged"
configuration on peer upstreams to extract the mesh gateway mode.
2023-01-18 13:43:53 -06:00
Derek Menteer 1f7e7abeac
Fix issue with incorrect proxycfg watch on upstream peer-targets. (#15865)
This fixes an issue where the incorrect partition was given to the
upstream target watch, which meant that failover logic would not
work correctly.
2023-01-03 10:44:08 -06:00
Nitya Dhanushkodi c7ef04c597
[OSS] extensions: refactor PluginConfiguration into a more generic type ExtensionConfiguration (#15846)
* extensions: refactor PluginConfiguration into a more generic type
ExtensionConfiguration

Also:
* adds endpoints configuration to lambda golden tests
* uses string constant for builtin/aws/lambda
Co-authored-by: Eric <eric@haberkorn.co>
2022-12-20 22:26:20 -08:00
Nitya Dhanushkodi d382ca0aec
extensions: refactor serverless plugin to use extensions from config entry fields (#15817)
docs: update config entry docs and the Lambda manual registration docs

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
Co-authored-by: Eric <eric@haberkorn.co>
2022-12-19 12:19:37 -08:00
cskh 04bf24c8c1
feat(ingress-gateway): support outlier detection of upstream service for ingress gateway (#15614)
* feat(ingress-gateway): support outlier detection of upstream service for ingress gateway

* changelog

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
2022-12-13 11:51:37 -05:00
Derek Menteer e87d35e313
Fix DialedDirectly configuration for Consul dataplane. (#15760)
Fix DialedDirectly configuration for Consul dataplane.
2022-12-13 09:16:31 -06:00
Derek Menteer 97ec5279aa
Fix local mesh gateway with peering discovery chains. (#15690)
Fix local mesh gateway with peering discovery chains.

Prior to this patch, discovery chains with peers would not
properly honor the mesh gateway mode for two reasons.

1. An incorrect target upstream ID was used to lookup the
mesh gateway mode. To fix this, the parent upstream uid is
now used instead of the discovery-chain-target-uid to find
the intended mesh gateway mode.

2. The watch for local mesh gateways was never initialized
for discovery chains. To fix this, the discovery chains are
now scanned, and a local GW watch is spawned if: the mesh
gateway mode is local and the target is a peering connection.
2022-12-07 13:07:42 -06:00
James Oulman 7e78fb7818
Add support for configuring Envoys route idle_timeout (#14340)
* Add idleTimeout

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-11-29 17:43:15 -05:00
Kyle Schochenmaier bf0f61a878
removes ioutil usage everywhere which was deprecated in go1.16 (#15297)
* update go version to 1.18 for api and sdk, go mod tidy
* removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward.
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2022-11-10 10:26:01 -06:00