github
/
consul
mirror of https://github.com/hashicorp/consul
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
consul/website/content/commands/connect/envoy.mdx

374 lines
17 KiB
Markdown
Raw Blame History

---
layout: commands
page_title: 'Commands: Connect Envoy'
sidebar_title: envoy
description: The connect envoy subcommand is used to generate a bootstrap configuration for Envoy.
---
# Consul Connect Envoy
Command: `consul connect envoy`
The connect Envoy command is used to generate a bootstrap configuration for
[Envoy proxy](https://envoyproxy.io) for use with [Consul
service mesh](/consul/docs/connect/).
Refer to the [examples](#examples) for guidance on common use cases,
such as [launching a service instance's sidecar proxy
when ACLs are enabled](#sidecar-proxy-with-acls-enabled).
The default behavior is to generate the necessary bootstrap configuration for
Envoy based on the environment variables and options provided and by talking to
the local Consul agent. It `exec`s an external Envoy binary with that
configuration leaving the Envoy process running in the foreground. An error is
returned on operating systems other than linux or macOS since Envoy does not
build for other platforms currently.
If the `-bootstrap` option is specified, the bootstrap config is generated in
the same way and then printed to stdout. This allows it to be redirected to a
file and used with `envoy --config-path bootstrap.json`. This works on all operating
systems allowing configuration to be generated on a host that Envoy doesn't
build on but then used in a virtualized environment that can run Envoy.
## Usage
Usage: `consul connect envoy [options] [-- pass-through options]`
#### Envoy Options for both Sidecars and Gateways
- `-proxy-id` - The [proxy service](/consul/docs/connect/registration/service-registration) ID.
This service ID must already be registered with the local agent unless a gateway is being
registered with the `-register` flag. As of Consul 1.8.0, this can also be
specified via the `CONNECT_PROXY_ID` environment variable.
- `-envoy-binary` - The full path to a specific Envoy binary to exec. By
default the current `$PATH` is searched for `envoy`.
- `-admin-bind` - The `host:port` to bind Envoy's admin HTTP API. Default is
`localhost:19000`. Envoy requires that this be enabled. The host part must be
resolvable DNS name or IP address.
- `-bootstrap` - If present, the command will simply output the generated
bootstrap config to stdout in JSON protobuf form. This can be directed to a
file and used to start Envoy with `envoy --config-path bootstrap.json`.
~> **Security Note:** If ACLs are enabled the bootstrap JSON will contain the
ACL token from `-token` or the environment and so should be handled as a secret.
This token grants the identity of any service it has `service:write` permission
for and so can be used to access any upstream service that that service is
allowed to access by [service mesh intentions](/consul/docs/connect/intentions).
- `-envoy-version` - The version of envoy that is being started. Default is
`1.23.1`. This is required so that the correct configuration can be generated.
- `-grpc-ca-file` - Path to a CA file to use for TLS when communicating with the
Consul agent through xDS. This can also be specified via the CONSUL_GRPC_CACERT
environment variable.
- `-grpc-ca-path` - Path to a directory of CA certificates to use for TLS when
communicating with the Consul agent through xDS. This can also be specified via
the CONSUL_GRPC_CAPATH environment variable.
- `-no-central-config` - By default the proxy's bootstrap configuration can be
customized centrally. This requires that the command run on the same agent
as the proxy will and that the agent is reachable when the command is run.
In cases where either assumption is violated this flag will prevent the
command attempting to resolve config from the local agent.
- `envoy_hcp_metrics_bind_socket_dir` - Specifies the directory where Envoy creates a unix socket.
Envoy sends metrics to the socket so that HCP collectors can connect to collect them."
The socket is not configured by default.
- `-envoy-ready-bind-address` - By default the proxy does not have a readiness probe
configured on it. This flag in conjunction with the `envoy-ready-bind-port` flag
configures where the envoy readiness probe is configured on the proxy. A `/ready` HTTP
endpoint will be instantiated at the specified address and port. When configured, Consul
uses `/ready` HTTP endpoints to check proxy health.
- `-envoy-ready-bind-port` - By default the proxy does not have a readiness probe
configured on it. This flag in conjunction with the `envoy-ready-bind-address` flag
configures where the envoy readiness probe is configured on the proxy. A `/ready` HTTP
endpoint will be instantiated at the specified address and port. When configured, Consul
uses `/ready` HTTP endpoints to check proxy health.
- `-prometheus-backend-port` - Sets the backend port for the "prometheus_backend"
cluster that `envoy_prometheus_bind_addr` will point to. Without this flag,
`envoy_prometheus_bind_addr` would point to the "self_admin" cluster where Envoy metrics
are exposed. The metrics merging feature in consul-k8s uses this to point to the
merged metrics endpoint combining Envoy and service metrics.
Only applicable when `envoy_prometheus_bind_addr` is set in proxy config.
- `-prometheus-scrape-path` - Sets the path where Envoy will expose metrics on the
`envoy_prometheus_bind_addr` listener. Default is `/metrics`. For example, if `envoy_prometheus_bind_addr`
is `0.0.0.0:20200`, and this flag is set to `/scrape-metrics`, prometheus metrics would
be scrapable at `0.0.0.0:20200/scrape-metrics`.
Only applicable when `envoy_prometheus_bind_addr` is set in proxy config.
- `-prometheus-ca-file` - Path to a CA file for Envoy to use when serving TLS on
the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr`
is set in proxy config.
- `-prometheus-ca-path` - Path to a directory of CA certificates for Envoy to use when
serving the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr`
is set in proxy config.
- `-prometheus-cert-file` - Path to a certificate file for Envoy to use when serving
TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr`
is set in proxy config.
- `-prometheus-key-file` - Path to a private key file for Envoy to use when serving
TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr`
is set in proxy config.
- `-ignore-envoy-compatibility` - If set to `true`, this flag ignores the Envoy version
compatibility check. We recommend setting this flag to `false` to ensure
compatibility with Envoy and prevent potential issues. Default is `false`.
- `-- [pass-through options]` - Any options given after a double dash are passed
directly through to the `envoy` invocation. See [Envoy's
documentation](https://www.envoyproxy.io/docs) for more details. The command
always specifies `--config-file` and `--v2-config-only` and by default passes
`--disable-hot-restart` see [hot restart](#envoy-hot-restart).
- `-enable-config-gen-logging` - This flag enables debug message logging when generating the Envoy bootstrap configuration to help troubleshoot issues. Logs output to `stderr`. For more information about generating the Envoy bootstrap configuration, refer to [Envoy proxy configuration](/consul/docs/connect/proxies/envoy#bootstrap-configuration).
#### Envoy Sidecar Proxy Options
- `-sidecar-for` - The _ID_ (not name if they differ) of the service instance
this proxy will represent. The target service doesn't need to exist on the
local agent yet but a [sidecar proxy
registration](/consul/docs/connect/registration/service-registration) with
`proxy.destination_service_id` equal to the passed value must be present. If
multiple proxy registrations targeting the same local service instance are
present the command will error and `-proxy-id` should be used instead.
As of Consul 1.8.0, this can also be specified via the `CONNECT_SIDECAR_FOR`
environment variable.
-> **Note:** If ACLs are enabled, a token granting `service:write` for the
_target_ service (configured in `proxy.destination_service_name`) must be
passed using the `-token` option or `CONSUL_HTTP_TOKEN` environment variable.
This token authorizes the proxy to obtain TLS certificates representing the
target service.
#### Envoy Gateway Options
- `-gateway` - Flag to indicate that Envoy should be configured as a Gateway.
Must be one of: `terminating`, `ingress`, or `mesh`.
If multiple gateways are managed by the same local agent then
`-proxy-id` should be used as well to specify the instance this represents.
- `-register` - Indicates that the gateway service should be registered
with the local agent instead of expecting it to already exist. This flag
is unused for traditional sidecar proxies.
- `-address` - The address to advertise for services within the local datacenter
to use to reach the gateway instance. This flag is used in combination with
`-register`. This takes the form of `<ip address>:<port>` but also supports go-sockaddr
templates.
If Envoy is configured as a terminating or mesh gateway, traffic from services
within the mesh will be received at the specified IP and port.
If Envoy is configured as an ingress gateway, a `/ready` HTTP endpoint will be
instantiated at the specified IP and port. Consul uses `/ready` HTTP endpoints
to check gateway health. The specified IP will also be used by the ingress
gateway when instantiating user-defined listeners configured in the
[ingress gateway](/consul/docs/connect/gateways/ingress-gateway) configuration entry.
~> **Note**: Ensure that user-defined ingress gateway listeners use a
different port than the port specified in `-address` so that they do not
conflict with the health check endpoint.
- `-admin-access-log-path` -
**Deprecated in Consul 1.15.0 in favor of [`proxy-defaults` access logs](/consul/docs/connect/config-entries/proxy-defaults#accesslogs).**
The path to write the access log for the administration
server. If no access log is desired specify `/dev/null`. By default it will
use `/dev/null`.
- `-bind-address` - The bind address to use instead of the default binding rules.
Specified as `<name>=<ip>:<port>` pairs. This flag may be used multiple times
to add multiple bind addresses.
- `-expose-servers` - Expose the servers for WAN federation via this mesh
gateway.
- `-wan-address` - The address to advertise for services within remote datacenters
to use to reach the gateway instance. This flag is used in combination with
`-register`. This takes the form of `<ip address>:<port>` but also supports go-sockaddr
templates.
- `-service` - The name of the gateway service to register. This flag is used
in combination with `-register`.
- `-deregister-after-critical` - The amount of time the gateway services health check can
be failing before being deregistered. This flag is used in combination with `-register`
#### Enterprise Options
@include 'http_api_partition_options.mdx'
@include 'http_api_namespace_options.mdx'
#### API Options
The standard API options are used to connect to the local agent to discover the
proxy configuration needed.
- `-grpc-addr=<addr>` - Address of the Consul agent with `grpc` port. This can
be an IP address or DNS address, but it must include the port. This can also
be specified via the CONSUL_GRPC_ADDR environment variable. In Consul 1.3 and
later, the default value is 127.0.0.1:8502, and https can optionally
be used instead. The scheme can also be set to HTTPS by setting the
environment variable CONSUL_HTTP_SSL=true. This may be a unix domain socket
using `unix:///path/to/socket` if the [agent is configured to
listen](/consul/docs/agent/config/config-files#addresses) that way.
-> **Note:** gRPC uses the same TLS
settings as the HTTPS API. If HTTPS is enabled then gRPC will require HTTPS
as well.
@include 'http_api_options_client.mdx'
## Examples
Assume a local service instance is registered on the local agent with a
sidecar proxy (using the [sidecar service
registration](/consul/docs/connect/registration/service-registration) helper) as below.
```hcl
service {
name = "web"
port = 8080
connect { sidecar_service {} }
}
```
### Basic sidecar proxy
The sidecar Envoy process can be started with.
```shell-session
$ consul connect envoy -sidecar-for web
```
This example assumes that the correct [environment variables](#api-options) are
used to set the local agent connection information and ACL token, or that the
agent is using all-default configuration.
### Sidecar proxy with ACLs enabled
In secure deployments, Consul's ACL system is enabled with a default `deny` policy.
To access Consul API resources, an API request must present a Consul ACL token
with the necessary privileges. If ACLs are enabled, you must provide an ACL token to the sidecar proxy. The token must grant the proxy privileges to register itself and the service it fronts and to access all potential upstreams of that service.
We recommend [using a service identity](/consul/commands/acl/token/create#create-a-token-for-a-service)
to directly create a token with the privileges necessary for a service and its sidecar proxy,
rather than creating a unique ACL policy for every service.
After creating a token using a service identity, provide the token when
launching the Envoy sidecar proxy instance with the `consul connect envoy` command.
You can provide the token through an environment variable or CLI flag.
Environment variables:
- [`CONSUL_HTTP_TOKEN`](/consul/commands#consul_http_token) or
- [`CONSUL_HTTP_TOKEN_FILE`](/consul/commands#consul_http_token_file)
CLI flags:
- [`-token`](/consul/commands/connect/envoy#token) or
- [`-token-file`](/consul/commands/connect/envoy#token-file)
```shell-session
$ export CONSUL_HTTP_TOKEN="<secret-id-of-token-with-service-identity-for-web>"
$ consul connect envoy -sidecar-for web
```
### Additional Envoy Arguments
To pass additional arguments directly to Envoy, for example output logging
level, you can use:
```shell-session
$ consul connect envoy -sidecar-for web -- --log-level debug
```
### Multiple Proxy Instances
To run multiple different proxy instances on the same host, you will
need to use `-admin-bind` on all but one to ensure they don't attempt to bind to
the same port as in the following example.
```shell-session
$ consul connect envoy -sidecar-for db -admin-bind localhost:19001
```
### Mesh Gateways
The mesh gateway Envoy process can be auto-registered and started with the following command.
```shell-session
$ consul connect envoy -gateway=mesh -register \
-address '{{ GetInterfaceIP "eth0" }}:8443' \
-wan-address '{{ GetInterfaceIP "eth1" }}:8443'
```
### Terminating Gateways
The terminating gateway Envoy process can be auto-registered and started with the following command.
```shell-session
$ consul connect envoy -gateway=terminating -register -service my-gateway \
-address '{{ GetInterfaceIP "eth0" }}:8443'
```
### Ingress Gateways
The ingress gateway Envoy process can be auto-registered and started with the following command.
```shell-session
$ consul connect envoy -gateway=ingress -register -service ingress-service \
-address '{{ GetInterfaceIP "eth0" }}:8888'
```
## Exec Security Details
The command needs to pass the bootstrap config through to Envoy. Envoy currently
only supports passing this as a file path or passing a whole string on the
command line with `--config-yaml`. Since the bootstrap needs to contain the ACL
token to authorize the proxy, this secret needs careful handling.
Passing a secret via command option is unacceptable as on many unix systems
these are readable to any user on the host for example via `/proc` or via a
setuid process like `ps`.
Creating a temporary file is more secure in that it can only be read by the
current user but risks leaving secret material on disk for an unbounded length
of time and in a location that is opaque to the operator.
To work around these issues, the command currently creates a temporary file and
immediately unlinks it so it can't be read by any other process that doesn't
already have the file descriptor. It then writes the bootstrap JSON, and unsets
the CLOEXEC bit on the file handle so that it remains available to the Envoy
process after exec. Finally it `exec`s Envoy with `--config-file /dev/fd/X`
where `X` is the the file descriptor number of the temp file.
This ensures that Envoy can read the file without any other normal user process
being able to (assuming they don't have privileged access to /proc). Once the
Envoy process stops, there is no longer any reference to the file to clean up.
## Envoy Hot Restart
Envoy supports hot restart which requires simple external coordination. By
default, this command will add `--disable-hot-restart` when it runs Envoy.
The reason for this default behavior is to make it easy to test and run local
demonstrations with multiple Envoy instances outside of cgroups or network
namespaces.
To use hot restart, Envoy needs to be started with either the `--restart-epoch`
option. If this command detects that option in the pass-through flags it will
_not_ add `--disable-hot-restart` allowing hot restart to work normally.
The only difference to note over running Envoy directly is that
`--restart-epoch` must be explicitly set to `0` for the initial launch of the
Envoy instance to avoid disabling hot restart entirely. The official
`hot-restarter.py` always sets this option so should work as recommended.
Reference in New Issue View Git Blame Copy Permalink