Commit Graph

17734 Commits (653b8c4f9d3748ef83f32ca795a53b171705317a)

Author SHA1 Message Date
Daniel Upton 653b8c4f9d proxycfg: server-local config entry data sources
This is the OSS portion of enterprise PR 2056.

This commit provides server-local implementations of the proxycfg.ConfigEntry
and proxycfg.ConfigEntryList interfaces, that source data from streaming events.

It makes use of the LocalMaterializer type introduced for peering replication,
adding the necessary support for authorization.

It also adds support for "wildcard" subscriptions (within a topic) to the event
publisher, as this is needed to fetch service-resolvers for all services when
configuring mesh gateways.

Currently, events will be emitted for just the ingress-gateway, service-resolver,
and mesh config entry types, as these are the only entries required by proxycfg
— the events will be emitted on topics named IngressGateway, ServiceResolver,
and MeshConfig topics respectively.

Though these events will only be consumed "locally" for now, they can also be
consumed via the gRPC endpoint (confirmed using grpcurl) so using them from
client agents should be a case of swapping the LocalMaterializer for an
RPCMaterializer.
2022-07-04 10:48:36 +01:00
Michael Klein 65b0faf96b
ui: allow searching services by admin-partition (#13650) 2022-06-30 17:24:52 +01:00
Michael Klein ef21100435
ui: peering chores (#13636)
* Update empty state topology downstreams to included peer info

* Add filter for filtering for service without ExternalSources
2022-06-30 15:47:04 +01:00
alex cd9ca4290a
peering: add imported/exported counts to peering (#13644)
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
2022-06-29 14:07:30 -07:00
Chris S. Kim b186731a2e
Fix ENT drift in files (#13647) 2022-06-29 16:53:22 -04:00
Matt Keeler 5105835cb2
Allow the /v1/internal/acl/authorize endpoint to authorize the “peering” resource (#13646)
Currently this just checks for operator read. In the near future it will check for peering specific rules once those are implemented.
2022-06-29 16:38:17 -04:00
Chris S. Kim d8b7940e40
Add internal endpoint to fetch peered upstream candidates from VirtualIP table (#13642)
For initial cluster peering TProxy support we consider all imported services of a partition to be potential upstreams.

We leverage the VirtualIP table because it stores plain service names (e.g. "api", not "api-sidecar-proxy").
2022-06-29 16:34:58 -04:00
Eric Haberkorn 653cb42944
Fix spelling mistake in serverless patcher (#13607)
passhthrough -> passthrough
2022-06-29 15:21:21 -04:00
David Yu 8552d875ae
docs: add controller to cluster peering docs (#13639)
* docs: add controller to cluster peering docs
2022-06-29 11:08:37 -07:00
John Cowen d66f5a6364
ui: Fix up peer ENT tests (#13633)
* ui: Add missing @nspaces

* Reorder peerings to be before any optionals

* Merge params instead of overwriting

* Reorder int tests
2022-06-29 19:07:39 +01:00
alex 07bc22e405
no 1.9 style metrics (#13532)
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>
2022-06-29 09:46:37 -07:00
alex beb8b03e8a
peering: reconcile/ hint active state for list (#13619)
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>
2022-06-29 09:43:50 -07:00
R.B. Boyer 31b95c747b
xds: modify rbac rules to use the XFCC header for peered L7 enforcement (#13629)
When the protocol is http-like, and an intention has a peered source
then the normal RBAC mTLS SAN field check is replaces with a joint combo
of:

    mTLS SAN field must be the service's local mesh gateway leaf cert
      AND
    the first XFCC header (from the MGW) must have a URI field that matches the original intention source

Also:

- Update the regex program limit to be much higher than the teeny
  defaults, since the RBAC regex constructions are more complicated now.

- Fix a few stray panics in xds generation.
2022-06-29 10:29:54 -05:00
Tu Nguyen 214495f2a2
Fix typo in cluster peering docs (#13574)
* Fix typo in cluster peering docs
* Remove highlight, update curly quotes
2022-06-28 15:54:57 -07:00
R.B. Boyer de0f9ac519
xds: have mesh gateways forward peered SpiffeIDs using the XFCC header (#13625) 2022-06-28 15:32:42 -05:00
R.B. Boyer 1a9c86ea8f
xds: mesh gateways now correctly load up peer-exported discovery chains using L7 protocols (#13624)
A mesh gateway will now configure the filter chains for L7 exported
services using the correct discovery chain information.
2022-06-28 14:52:25 -05:00
R.B. Boyer 7133ee38f6
test: for upgrade compatibility tests retain assigned container ip addresses on upgrade (#13615)
Use a synthetic pod construct to hold onto the IP address in the
interim.
2022-06-28 09:50:13 -05:00
Dan Upton ebf74d08fd
test: run Envoy integration tests against both servers and clients (#13610) 2022-06-28 13:15:45 +01:00
Michele Degges 977c6e58de
Turn off sec-scanner check (#13614) 2022-06-27 15:52:51 -07:00
Evan Culver d4fdddf8d4
Fix verifications by using updated arm package names (#13601)
Co-authored-by: alex <8968914+acpana@users.noreply.github.com>
2022-06-27 14:00:27 -07:00
R.B. Boyer 0fa828db76
peering: replicate all SpiffeID values necessary for the importing side to do SAN validation (#13612)
When traversing an exported peered service, the discovery chain
evaluation at the other side may re-route the request to a variety of
endpoints. Furthermore we intend to terminate mTLS at the mesh gateway
for arriving peered traffic that is http-like (L7), so the caller needs
to know the mesh gateway's SpiffeID in that case as well.

The following new SpiffeID values will be shipped back in the peerstream
replication:

- tcp: all possible SpiffeIDs resulting from the service-resolver
        component of the exported discovery chain

- http-like: the SpiffeID of the mesh gateway
2022-06-27 14:37:18 -05:00
Kyle Havlovitz 162caaecb3
Merge pull request #13611 from hashicorp/prometheus-tls-docs
Update docs for prometheus TLS options
2022-06-27 09:51:06 -07:00
Kyle Havlovitz 340a234361 Update docs for prometheus TLS options 2022-06-27 09:33:27 -07:00
Amier Chery aa508c8a42
Merge pull request #13516 from maxb/docs-fix-metric-dots
Fix use of trailing dots on metric names in telemetry.mdx
2022-06-27 10:31:11 -04:00
Amier Chery b429d00719
Merge pull request #13603 from loicsaintroch/patch-1
Add HashiBox to community tools
2022-06-27 10:29:30 -04:00
Loïc Saint-Roch 1740181f61
Add HashiBox to community tools 2022-06-26 15:50:25 +02:00
Kyle Havlovitz 22a5532aef
Merge pull request #13481 from hashicorp/envoy-prometheus-tls
Add TLS support in Envoy Prometheus endpoint
2022-06-24 15:36:40 -07:00
alex 53f0cf5835
peering, internal: support UIServices, UINodes, UINodeInfo (#13577) 2022-06-24 15:17:35 -07:00
Michele Degges 0b3f90c3e7
[CI-only] Dev tag update for main (#13541) 2022-06-24 13:45:57 -07:00
Evan Culver 359328dacb
Remove trigger-oss-merge job (#13600) 2022-06-24 13:45:30 -07:00
Chris S. Kim 2e4cb6f77d
Add new index for PeeredServiceName and ServiceVirtualIP (#13582)
For TProxy we will be leveraging the VirtualIP table, which needs to become peer-aware
2022-06-24 14:38:39 -04:00
R.B. Boyer a1e911a70c
tests: ensure integration tests show logs from the containers to help debugging (#13593) 2022-06-24 10:26:17 -05:00
Matt Keeler a3a4495e78
Clarify the wording of the peering limitations in the preview (#13590) 2022-06-24 09:58:31 -04:00
Frank DiRocco 4a4a64446f
update terraform module location for consul aws modules (#13522)
Co-authored-by: Paul Glass <pglass@hashicorp.com>
2022-06-23 22:10:44 -07:00
Paul Glass 02ee123b7b
docs: Update ECS docs for IAM auth method support (#13222) 2022-06-23 16:42:40 -05:00
David Yu ea0966ff5e
docs: add missing $ gossip key rotation (#13581) 2022-06-23 14:31:05 -07:00
David Yu 4d9922c1e4
docs: add indent to code block config tab to align with other branches (#13573) 2022-06-23 08:38:36 -07:00
alex 20ecf0febd
Merge pull request #13570 from hashicorp/acpance/peering-oss-intentions
oss: peering, http: get peer service intentions (#2098)
2022-06-23 08:15:59 -07:00
Will Jordan 34ecbc1d71
Add per-node max indexes (#12399)
Adds fine-grained node.[node] entries to the index table, allowing blocking queries to return fine-grained indexes that prevent them from returning immediately when unrelated nodes/services are updated.

Co-authored-by: kisunji <ckim@hashicorp.com>
2022-06-23 11:13:25 -04:00
Chris S. Kim ba89a7d9b0
Make memdb indexers generic (#13558)
We have many indexer functions in Consul which take interface{} and type assert before building the index. We can use generics to get rid of the initial plumbing and pass around functions with better defined signatures. This has two benefits: 1) Less verbosity; 2) Developers can parse the argument types to memdb schemas without having to introspect the function for the type assertion.
2022-06-23 11:07:19 -04:00
Matt Keeler 7a4d13b0b2
Port over the index 0 -> 1 code that lived in the old rpc setQueryMeta function. (#13561) 2022-06-23 09:34:47 -04:00
Michael Klein 5de75550d3
ui: feature-flagged peering mvp (#13425)
* add peers route

* add peers to nav

* use regular app ui patterns peers template

* use empty state in peers UI

* mock `v1/peerings` request

* implement custom adapter/serializer for `peers`-model

* index request for peerings on peers route

* update peers list to show as proper list

* Use tailwind for easier styling

* Unique ids in peerings response mock-api

* Add styling peerings list

* Allow creating empty tooltip

To make it easier to iterate over a set of items where some items
should not display a tooltip and others should.

* Add tooltip Peerings:Badge

* Add undefined peering state badge

* Remove imported/exported services count peering

This won't be included in the initial version of the API response

* Implement Peerings::Search

* Make it possible to filter peerings by name

* Install ember-keyboard

For idiomatic handling of key-presses.

* Clear peering search input when pressing `Escape`

* use peers.index instead of peers for peerings listing

* Allow to include peered services in services-query

* update services mock to add peerName

* add Consul::Peer component

To surface peering information on a resource

* add PeerName as attribute to service model

* surface peering information in service list

* Add tooltip to Consul::Peer

* Make services searchable by peer-name

* Allow passing optional query-params to href-to

* Add peer query-param to dc.services.show

* Pass peer as query-param services listing

* support option peer route-param

* set peer-name undefined in services serializer when empty

* update peer route-param when navigating to peered service

* request sercice with peer-name if need be

* make sure to reset peer route-param when leaving service.show

* componentize services.peer-info

* surface peer info services.show

* make sure to reset peer route-param in main nav

* fix services breadcrumb services.intentions

we need to reset peer route-param here to not break the app

* surface peer when querying for it on service api call

* query for peer info service-instance api calls

* surface peer info service-instance.show

* Camelize peer attributes to match rest of app

* Refactor peers.index to reflect camelized attributes for peer

* Remove unused query-params services.show

* make logo href reset peer route-param

* Cleanup optional peer param query service-instance

* Use replace decorator instead of serializer for empty peerName

* make sure to only send peer info when correct qp is passed

* Always send qp for querying peers services request

* rename with-imports to with-peers

* Use css for peer-icon

* Refactor bucket-list component to surface peer-info

* Remove Consul::Peer component

This info is now displayed via the bucket-list component

* Fix bucket-list component to surface service again

* Update bucket-list docs to reflect peer-info addition

* Remove tailwind related styles

* Remove consul-tailwind package

We won't be using tailwind for now

* Fix typo badge scss

* Add with-import handling mock-api nodes

* Add peerName to node attributes

* include peers when querying nodes

* reflect api updates node list mock

* Create consul::node::peer-info component

* Surface peer-info in nodes list

* Mock peer response for node request

* Make it possible to add peer-name to node request

* Update peer route-param when linking to node

* Reset peers route-param when leaving nodes.show

We need to reset the route-param to not introduce a bug - otherwise
subsequent node show request would request with the old peer query-param

* Add sourcePeer intentions api mock

* add SourcePeer attr to intentions model

* Surface peering info on intentions list

* Request peered intentions differently intentions.edit

* Handle peer info in intentions/exact mock

* Surface peering info intention view

* Add randomized peer data topology mock

* Surface peer info topology view

* fix service/peer-info styling

We aren't using tailwind anymore - we need to create a custom scss file

* Update peerings api mocks

* Update peerings::badge with updated styling

* cleanup intentions/exact mock

* Create watcher component to declaratively register polling

* Poll peers in background when on peers route

* use existing colors for peering-badge

* Add test for requesting service with `with-peers`-query

* add imported/exported count to peers model

* update mock-api to surface exported/imported count on peers

* Show exported/imported peers count on peers list

* Use translations for service import/export UI peers

* Make sure to ask for nodes with peers

* Add match-url step for easier url testing of service urls

* Add test for peer-name on peered services

* Add test for service navigation peered service

* Implement feature-flag handling

* Enable peering feature in test and development

* Redirect peers to services.index when feature-flag is disabled

* Only query for peers when feature is enabled

* Only show peers in nav when feature is enabled

* Componentize peering service count detail

* Handle non-state Peerings::Badge

* Use Peerings::ServiceCount in peerings list

* Only send peer query for peered service-instances.

* Add step to visit url directly

* add test for accessing peered service directly

* Remove unused service import peers.index

* Only query for peer when peer provided node-adapter

* fix tests
2022-06-23 14:16:26 +01:00
David Yu a0b94d9a3a
docs: add Core requirements to cluster peering k8s docs (#13569)
* docs: add Core requirements to cluster peering k8s docs

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2022-06-22 19:12:08 -07:00
acpana 99c2e11328
oss: peering, http: get peer service intentions (#2098)
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>
2022-06-22 16:25:09 -07:00
trujillo-adam e00ebbb87d
Merge pull request #13492 from hashicorp/docs-ecs-mesh-gw
Docs for ECS Mesh Gateway
2022-06-22 15:55:31 -07:00
Kyle Schochenmaier f961c8610b
[docs] update doc headers (#13527)
* update helm docs to have correct headers
2022-06-22 15:56:25 -05:00
Kyle Havlovitz 6c85770ff8
Merge pull request #13526 from hashicorp/dns-parititon-docs
docs: Clarify section on partitioned node DNS lookups
2022-06-22 10:59:04 -07:00
Kyle Havlovitz 0cfe43f594 docs: Clarify section on partitioned node DNS lookups 2022-06-22 10:41:13 -07:00
trujillo-adam c100e4cdb1 applied feedback from review 2022-06-22 10:18:56 -07:00
Tu Nguyen 878d19c3d0
Merge pull request #13550 from hashicorp/docs/peering-upstream-annotation
Docs/peering upstream annotation
2022-06-22 01:02:23 -07:00