consul/website/content/docs/connect/native/index.mdx

159 lines
8.0 KiB
Plaintext
Raw Normal View History

---
2020-04-07 18:55:19 +00:00
layout: docs
2022-09-16 15:28:32 +00:00
page_title: Service Mesh Native App Integration - Overview
2020-04-07 18:55:19 +00:00
description: >-
When using sidecar proxies is not possible, applications can natively integrate with Consul service mesh, but have reduced access to service mesh features. Learn how "mesh-native" or "connect-native" apps use mTLS to authenticate with Consul and how to add integrations to service registrations.
---
2022-09-13 20:24:43 +00:00
# Service Mesh Native App Integration Overview
~> **Note:** The Native App Integration does not support many of the Consul's service
mesh features, and is not under active development.
The [Envoy proxy](/consul/docs/connect/proxies/envoy) should be used for most production
environments.
Applications can natively integrate with Consul's service mesh API to support accepting
and establishing connections to other mesh services without the overhead of a
[proxy sidecar](/consul/docs/connect/proxies). This option is especially useful
2020-04-06 20:27:35 +00:00
for applications that may be experiencing performance issues with the proxy
sidecar deployment. This page will cover the high-level overview of
integration, registering the service, etc. For language-specific examples, see
[docs] Reorganize connect documentation for clarity (#5864) * clarify possibilities for centralized proxy configuration * add line breaks to config entries file * add info about centralized config to built in proxy doc * mondify connect landing page to help with navigation * move internals details to its own page * link fixes and shortening text on main page * put built-in proxy options on its own page * add configuration details for connect * clarify security title and add observability page * reorganize menu * remove observability from configuration section * Update website/source/docs/connect/configuration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/index.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/agent/config_entries.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/configuration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * rename connect section to include service mesh * reorganize sections per suggestions from paul * add configuration edits from paul * add internals edits from paul * add observability edits from paul * reorganize pages and menu * Update website/source/docs/connect/configuration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * menu corrections and edits * incorporate some of pauls comments * incorporate more of pauls comments * Update website/source/docs/connect/configuration.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Update website/source/docs/connect/index.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Update website/source/docs/connect/index.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Update website/source/docs/connect/registration.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * incorporate kaitlin and pavanni feedback * add redirect * fix conflicts in index file * Resolve conflicts in index file * correct links for new organization * Update website/source/docs/connect/proxies.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/registration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/registration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/registration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * add title to service registration page
2019-06-14 05:52:50 +00:00
the sidebar navigation to the left. It is also required if your service uses
relies on a dynamic set of upstream services.
2018-06-06 05:42:59 +00:00
Service mesh traffic is just basic mutual TLS. This means that almost any application
can easily integrate with Consul service mesh. There is no custom protocol in use;
any language that supports TLS can accept and establish mesh-based
2018-06-06 05:42:59 +00:00
connections.
We currently provide an easy-to-use [Go integration](/consul/docs/connect/native/go)
2018-06-06 18:44:10 +00:00
to assist with the getting the proper certificates, verifying connections,
etc. We plan to add helper libraries for other languages in the future.
However, without library support, it is still possible for any major language
to integrate with Consul service mesh.
The noun _connect_ is used throughout this documentation to refer to the connect
subsystem that provides Consul's service mesh capabilities.
2018-06-06 18:44:10 +00:00
2018-06-06 05:42:59 +00:00
## Overview
The primary work involved in natively integrating with service mesh is
[acquiring the proper TLS certificate](/consul/api-docs/agent/connect#service-leaf-certificate),
[verifying TLS certificates](/consul/api-docs/agent/connect#certificate-authority-ca-roots),
and [authorizing inbound connections or requests](/consul/api-docs/connect/intentions#list-matching-intentions).
2018-06-06 18:44:10 +00:00
All of this is done using the Consul HTTP APIs linked above.
2018-06-06 05:42:59 +00:00
An overview of the sequence is shown below. The diagram and the following
details may seem complex, but this is a _regular mutual TLS connection_ with
an API call to verify the incoming client certificate.
![Native Integration Overview](/img/connect-native-overview.png)
2018-06-06 05:42:59 +00:00
-> **Note:** This diagram depicts the simpler networking layer 4 (e.g. TCP) [integration
mechanism](/consul/api-docs/agent/connect#authorize).
2018-06-06 05:42:59 +00:00
Details on the steps are below:
2020-04-06 20:27:35 +00:00
- **Service discovery** - This is normal service discovery using Consul,
a static IP, or any other mechanism. If you're using Consul DNS, the
Docs/services refactor docs day 122022 (#16103) * converted main services page to services overview page * set up services usage dirs * added Define Services usage page * converted health checks everything page to Define Health Checks usage page * added Register Services and Nodes usage page * converted Query with DNS to Discover Services and Nodes Overview page * added Configure DNS Behavior usage page * added Enable Static DNS Lookups usage page * added the Enable Dynamic Queries DNS Queries usage page * added the Configuration dir and overview page - may not need the overview, tho * fixed the nav from previous commit * added the Services Configuration Reference page * added Health Checks Configuration Reference page * updated service defaults configuraiton entry to new configuration ref format * fixed some bad links found by checker * more bad links found by checker * another bad link found by checker * converted main services page to services overview page * set up services usage dirs * added Define Services usage page * converted health checks everything page to Define Health Checks usage page * added Register Services and Nodes usage page * converted Query with DNS to Discover Services and Nodes Overview page * added Configure DNS Behavior usage page * added Enable Static DNS Lookups usage page * added the Enable Dynamic Queries DNS Queries usage page * added the Configuration dir and overview page - may not need the overview, tho * fixed the nav from previous commit * added the Services Configuration Reference page * added Health Checks Configuration Reference page * updated service defaults configuraiton entry to new configuration ref format * fixed some bad links found by checker * more bad links found by checker * another bad link found by checker * fixed cross-links between new topics * updated links to the new services pages * fixed bad links in scale file * tweaks to titles and phrasing * fixed typo in checks.mdx * started updating the conf ref to latest template * update SD conf ref to match latest CT standard * Apply suggestions from code review Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> * remove previous version of the checks page * fixed cross-links * Apply suggestions from code review Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> --------- Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
2023-02-28 22:09:56 +00:00
[`<service>.connect`](/consul/docs/services/discovery/dns-static-lookups#service-mesh-enabled-service-lookups)
syntax to find mesh-capable endpoints for a service. After service
2020-04-06 20:27:35 +00:00
discovery, choose one address from the list of **service addresses**.
- **Mutual TLS** - As a client, connect to the discovered service address
over normal TLS. As part of the TLS connection, provide the
[service certificate](/consul/api-docs/agent/connect#service-leaf-certificate)
2020-04-06 20:27:35 +00:00
as the client certificate. Verify the remote certificate against the
[public CA roots](/consul/api-docs/agent/connect#certificate-authority-ca-roots).
2020-04-06 20:27:35 +00:00
As a client, if the connection is established then you've established
a mesh-based connection and there are no further steps!
2020-04-06 20:27:35 +00:00
- **Authorization** - As a server accepting connections, verify the client
certificate against the [public CA
roots](/consul/api-docs/agent/connect#certificate-authority-ca-roots). After verifying
the certificate, parse some basic fields from it and use those to determine
if the connection should be allowed. How this is done is dependent on
the level of integration desired:
- **Simple integration (TCP-only)** - Call the [authorizing
API](/consul/api-docs/agent/connect#authorize) against the local agent. If this returns
successfully, complete the TLS handshake and establish the connection. If
authorization fails, close the connection.
-> **NOTE:** This API call is expected to be called in the connection path,
so if the local Consul agent is down or unresponsive it will effect the
success rate of new connections. The agent uses locally cached data to
authorize the connection and typically responds in microseconds. Therefore,
the impact to the TLS handshake is typically microseconds.
- **Complete integration** - Like how the calls to acquire the leaf
certificate and CA roots are expected to be done out of band and reused, so
should the [intention match
API](/consul/api-docs/connect/intentions#list-matching-intentions). With all of the
relevant intentions cached for the destination, all enforcement operations
can be done entirely by the service without calling any Consul APIs in the
connection or request path. If the service is networking layer 7 (e.g.
HTTP) aware it can safely enforce intentions per _request_ instead of the
coarser per _connection_ model.
2018-06-06 05:42:59 +00:00
Docs/services refactor docs day 122022 (#16103) * converted main services page to services overview page * set up services usage dirs * added Define Services usage page * converted health checks everything page to Define Health Checks usage page * added Register Services and Nodes usage page * converted Query with DNS to Discover Services and Nodes Overview page * added Configure DNS Behavior usage page * added Enable Static DNS Lookups usage page * added the Enable Dynamic Queries DNS Queries usage page * added the Configuration dir and overview page - may not need the overview, tho * fixed the nav from previous commit * added the Services Configuration Reference page * added Health Checks Configuration Reference page * updated service defaults configuraiton entry to new configuration ref format * fixed some bad links found by checker * more bad links found by checker * another bad link found by checker * converted main services page to services overview page * set up services usage dirs * added Define Services usage page * converted health checks everything page to Define Health Checks usage page * added Register Services and Nodes usage page * converted Query with DNS to Discover Services and Nodes Overview page * added Configure DNS Behavior usage page * added Enable Static DNS Lookups usage page * added the Enable Dynamic Queries DNS Queries usage page * added the Configuration dir and overview page - may not need the overview, tho * fixed the nav from previous commit * added the Services Configuration Reference page * added Health Checks Configuration Reference page * updated service defaults configuraiton entry to new configuration ref format * fixed some bad links found by checker * more bad links found by checker * another bad link found by checker * fixed cross-links between new topics * updated links to the new services pages * fixed bad links in scale file * tweaks to titles and phrasing * fixed typo in checks.mdx * started updating the conf ref to latest template * update SD conf ref to match latest CT standard * Apply suggestions from code review Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> * remove previous version of the checks page * fixed cross-links * Apply suggestions from code review Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> --------- Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
2023-02-28 22:09:56 +00:00
## Update certificates and certificate roots
2018-06-06 05:42:59 +00:00
The leaf certificate and CA roots can be updated at any time and the
natively integrated application must react to this relatively quickly
so that new connections are not disrupted. This can be done through
Consul blocking queries (HTTP long polling) or through periodic polling.
The API calls for
[acquiring a service mesh TLS certificate](/consul/api-docs/agent/connect#service-leaf-certificate)
and [reading service mesh CA roots](/consul/api-docs/agent/connect#certificate-authority-ca-roots)
2018-06-06 05:42:59 +00:00
both support
[blocking queries](/consul/api-docs/features/blocking). By using blocking
2018-06-06 05:42:59 +00:00
queries, an application can efficiently wait for an updated value. For example,
the leaf certificate API will block until the certificate is near expiration
or the signing certificates have changed and will issue and return a new
certificate.
In some languages, using blocking queries may not be simple. In that case,
we still recommend using the blocking query parameters but with a very short
`timeout` value set. Doing this is documented with
[blocking queries](/consul/api-docs/features/blocking). The low timeout will
2018-06-06 05:42:59 +00:00
ensure the API responds quickly. We recommend that applications poll the
certificate endpoints frequently, such as multiple times per minute.
The overhead for the blocking queries (long or periodic polling) is minimal.
The API calls are to the local agent and the local agent uses locally
cached data multiplexed over a single TCP connection to the Consul leader.
Even if a single machine has 1,000 mesh-enabled services all blocking
2018-06-06 05:42:59 +00:00
on certificate updates, this translates to only one TCP connection to the
Consul server.
Some language libraries such as the
[Go library](/consul/docs/connect/native/go) automatically handle updating
2018-06-06 05:42:59 +00:00
and locally caching the certificates.
Docs/services refactor docs day 122022 (#16103) * converted main services page to services overview page * set up services usage dirs * added Define Services usage page * converted health checks everything page to Define Health Checks usage page * added Register Services and Nodes usage page * converted Query with DNS to Discover Services and Nodes Overview page * added Configure DNS Behavior usage page * added Enable Static DNS Lookups usage page * added the Enable Dynamic Queries DNS Queries usage page * added the Configuration dir and overview page - may not need the overview, tho * fixed the nav from previous commit * added the Services Configuration Reference page * added Health Checks Configuration Reference page * updated service defaults configuraiton entry to new configuration ref format * fixed some bad links found by checker * more bad links found by checker * another bad link found by checker * converted main services page to services overview page * set up services usage dirs * added Define Services usage page * converted health checks everything page to Define Health Checks usage page * added Register Services and Nodes usage page * converted Query with DNS to Discover Services and Nodes Overview page * added Configure DNS Behavior usage page * added Enable Static DNS Lookups usage page * added the Enable Dynamic Queries DNS Queries usage page * added the Configuration dir and overview page - may not need the overview, tho * fixed the nav from previous commit * added the Services Configuration Reference page * added Health Checks Configuration Reference page * updated service defaults configuraiton entry to new configuration ref format * fixed some bad links found by checker * more bad links found by checker * another bad link found by checker * fixed cross-links between new topics * updated links to the new services pages * fixed bad links in scale file * tweaks to titles and phrasing * fixed typo in checks.mdx * started updating the conf ref to latest template * update SD conf ref to match latest CT standard * Apply suggestions from code review Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> * remove previous version of the checks page * fixed cross-links * Apply suggestions from code review Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> --------- Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
2023-02-28 22:09:56 +00:00
## Service registration
2018-06-06 05:42:59 +00:00
Mesh-native applications must tell Consul that they support service mesh
2018-06-06 05:42:59 +00:00
natively. This enables the service to be returned as part of service
discovery for service mesh-capable services used by other mesh-native applications
and client [proxies](/consul/docs/connect/proxies).
2018-06-06 05:42:59 +00:00
Docs/services refactor docs day 122022 (#16103) * converted main services page to services overview page * set up services usage dirs * added Define Services usage page * converted health checks everything page to Define Health Checks usage page * added Register Services and Nodes usage page * converted Query with DNS to Discover Services and Nodes Overview page * added Configure DNS Behavior usage page * added Enable Static DNS Lookups usage page * added the Enable Dynamic Queries DNS Queries usage page * added the Configuration dir and overview page - may not need the overview, tho * fixed the nav from previous commit * added the Services Configuration Reference page * added Health Checks Configuration Reference page * updated service defaults configuraiton entry to new configuration ref format * fixed some bad links found by checker * more bad links found by checker * another bad link found by checker * converted main services page to services overview page * set up services usage dirs * added Define Services usage page * converted health checks everything page to Define Health Checks usage page * added Register Services and Nodes usage page * converted Query with DNS to Discover Services and Nodes Overview page * added Configure DNS Behavior usage page * added Enable Static DNS Lookups usage page * added the Enable Dynamic Queries DNS Queries usage page * added the Configuration dir and overview page - may not need the overview, tho * fixed the nav from previous commit * added the Services Configuration Reference page * added Health Checks Configuration Reference page * updated service defaults configuraiton entry to new configuration ref format * fixed some bad links found by checker * more bad links found by checker * another bad link found by checker * fixed cross-links between new topics * updated links to the new services pages * fixed bad links in scale file * tweaks to titles and phrasing * fixed typo in checks.mdx * started updating the conf ref to latest template * update SD conf ref to match latest CT standard * Apply suggestions from code review Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> * remove previous version of the checks page * fixed cross-links * Apply suggestions from code review Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com> --------- Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
2023-02-28 22:09:56 +00:00
You can enable native service mesh support directly in the [service definition](/consul/docs/services/configuration/services-configuration-reference#connect) by configuring the `connect` block. In the following example, the `redis` service is configured to support service mesh natively:
2018-06-06 05:42:59 +00:00
```json
{
"service": {
"name": "redis",
"port": 8000,
"connect": {
"native": true
}
}
}
```
Services that support service mesh natively are still returned through the standard
service discovery mechanisms in addition to the mesh-only service discovery
2018-06-06 05:42:59 +00:00
mechanisms.