Browse Source

new docs for consul and consul-k8s troubleshoot command (#16284)

* new docs for consul and consul-k8s troubleshoot command

* add changelog

* add troubleshoot command

* address comments, and update cli output to match

* revert changes to troubleshoot upstreams, changes will happen in separate pr

* Update .changelog/16284.txt

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>

* address comments

* update trouble proxy output

* add missing s, add required fields in usage

---------

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
pull/16322/head
malizz 2 years ago committed by GitHub
parent
commit
c9c49ea3a2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 3
      .changelog/16284.txt
  2. 1
      website/content/commands/index.mdx
  3. 31
      website/content/commands/troubleshoot/index.mdx
  4. 68
      website/content/commands/troubleshoot/proxy.mdx
  5. 37
      website/content/commands/troubleshoot/upstreams.mdx
  6. 101
      website/content/docs/k8s/k8s-cli.mdx
  7. 17
      website/data/commands-nav-data.json

3
.changelog/16284.txt

@ -0,0 +1,3 @@
```release-note:feature
cli: adds new CLI commands `consul troubleshoot upstreams` and `consul troubleshoot proxy` to troubleshoot Consul's service mesh configuration and network issues.
```

1
website/content/commands/index.mdx

@ -56,6 +56,7 @@ Available commands are:
services Interact with services
snapshot Saves, restores and inspects snapshots of Consul server state
tls Builtin helpers for creating CAs and certificates
troubleshoot Provides tools to troubleshoot Consul's service mesh configuration
validate Validate config files/directories
version Prints the Consul version
watch Watch for changes in Consul

31
website/content/commands/troubleshoot/index.mdx

@ -0,0 +1,31 @@
---
layout: commands
page_title: 'Commands: Troubleshoot'
description: >-
The `consul troubleshoot` command provides tools to troubleshoot Consul's service mesh configuration.
---
# Consul Troubleshooting
Command: `consul troubleshoot`
Use the `troubleshoot` command to diagnose Consul service mesh configuration or network issues.
## Usage
```text
Usage: consul troubleshoot <subcommand> [options]
# ...
Subcommands:
proxy Troubleshoots service mesh issues from the current Envoy instance
upstreams Gets upstream Envoy identifiers and IPs configured for the proxy
```
For more information, examples, and usage about a subcommand, click on the name
of the subcommand in the sidebar or one of the links below:
- [proxy](/consul/commands/troubleshoot/proxy)
- [upstreams](/consul/commands/troubleshoot/upstreams)

68
website/content/commands/troubleshoot/proxy.mdx

@ -0,0 +1,68 @@
---
layout: commands
page_title: 'Commands: Troubleshoot Proxy'
description: >-
The `consul troubleshoot proxy` diagnoses Consul service mesh configuration and network issues to an upstream.
---
# Consul Troubleshoot Proxy
Command: `consul troubleshoot proxy`
The `troubleshoot proxy` command diagnoses Consul service mesh configuration and network issues to an upstream.
## Usage
Usage: `consul troubleshoot proxy (-upstream-ip <ip>|-upstream-envoy-id <envoy-id>) [options]`
This command requires `-envoy-admin-endpoint` or `-upstream-ip` to specify the upstream.
#### Command Options
- `-envoy-admin-endpoint=<value>` - Envoy admin endpoint address for the local Envoy instance.
Defaults to `127.0.0.1:19000`.
- `-upstream-ip=<value>` - The IP address of the upstream service; Use when the upstream is reached via a transparent proxy DNS address (KubeDNS or Consul DNS)
- `-upstream-envoy-id=<value>` - The Envoy identifier of the upstream service; Use when the upstream is explicitly configured on the downstream service.
## Examples
The following example illustrates how to troubleshoot Consul service mesh configuration and network issues to an upstream from a source proxy.
```shell-session
$ consul troubleshoot proxy -upstream-ip 10.4.6.160
==> Validation
✓ Certificates are valid
✓ Envoy has 0 rejected configurations
✓ Envoy has detected 0 connection failure(s)
✓ Listener for upstream "backend" found
✓ Route for upstream "backend" found
✓ Cluster "backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
✓ Healthy endpoints for cluster "backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
✓ Cluster "backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
! No healthy endpoints for cluster "backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
-> Check that your upstream service is healthy and running
-> Check that your upstream service is registered with Consul
-> Check that the upstream proxy is healthy and running
-> If you are explicitly configuring upstreams, ensure the name of the upstream is correct
```
The following example troubleshoots Consul service mesh configuration and network issues from the local Envoy instance to the given upstream.
```shell-session
$ consul-k8s troubleshoot proxy -upstream-envoy-id db
==> Validation
✓ Certificates are valid
✓ Envoy has 0 rejected configurations
✓ Envoy has detected 0 connection failure(s)
✓ Listener for upstream "db" found
✓ Route for upstream "db" found
✓ Cluster "db.default.dc1.internal.e08fa6d6-e91e-dfe0.consul" for upstream "db" found
✓ Healthy endpoints for cluster "db.default.dc1.internal.e08fa6d6-e91e-dfe0.consul" for upstream "db" found
If you are still experiencing issues, you can:
-> Check intentions to ensure the upstream allows traffic from this source
-> If using transparent proxy, ensure DNS resolution is to the same IP you have verified here
```

37
website/content/commands/troubleshoot/upstreams.mdx

@ -0,0 +1,37 @@
---
layout: commands
page_title: 'Commands: Troubleshoot Upstreams'
description: >-
The `consul troubleshoot upstreams` lists the available upstreams in the Consul service mesh from the current service.
---
# Consul Troubleshoot Upstreams
Command: `consul troubleshoot upstreams`
The `troubleshoot upstreams` lists the available upstreams in the Consul service mesh from the current service.
## Usage
Usage: `consul troubleshoot upstreams [options]`
#### Command Options
- `-envoy-admin-endpoint=<value>` - Envoy admin endpoint address for the local Envoy instance.
Defaults to `127.0.0.1:19000`.
## Examples
Display all transparent proxy upstreams in Consul service mesh from the current Envoy instance.
```shell-session
$ consul troubleshoot upstreams
==> Upstreams (explicit upstreams only) (0)
==> Upstreams IPs (transparent proxy only) (1)
[10.4.6.160 240.0.0.3] true map[backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul]
If you cannot find the upstream address or cluster for a transparent proxy upstream:
- Check intentions: Tproxy upstreams are configured based on intentions. Make sure you have configured intentions to allow traffic to your upstream.
- To check that the right cluster is being dialed, run a DNS lookup for the upstream you are dialing. For example, run `dig backend.svc.consul` to return the IP address for the `backend` service. If the address you get from that is missing from the upstream IPs, it means that your proxy may be misconfigured.
```

101
website/content/docs/k8s/k8s-cli.mdx

@ -33,6 +33,7 @@ You can use the following commands with `consul-k8s`.
- [`proxy list`](#proxy-list): List all Pods running proxies managed by Consul.
- [`proxy read`](#proxy-read): Inspect the Envoy configuration for a given Pod.
- [`status`](#status): Check the status of a Consul installation on Kubernetes.
- [`troubleshoot`](#troubleshoot): Troubleshoot Consul service mesh and networking issues from a given pod.
- [`uninstall`](#uninstall): Uninstall Consul deployment.
- [`upgrade`](#upgrade): Upgrade Consul on Kubernetes from an existing installation.
- [`version`](#version): Print the version of the Consul on Kubernetes CLI.
@ -490,6 +491,106 @@ $ consul-k8s status
✓ Consul clients healthy (3/3)
```
### `troubleshoot`
The `troubleshoot` command exposes two subcommands for troubleshooting Consul
service mesh and network issues from a given pod.
- [`troubleshoot upstreams`](#troubleshoot-upstreams): List all Envoy upstreams in Consul service mesh from the given pod.
- [`troubleshoot proxy`](#troubleshoot-proxy): Troubleshoot Consul service mesh configuration and network issues between the given pod and the given upstream.
### `troubleshoot upstreams`
```shell-session
$ consul-k8s troubleshoot upstreams -pod <pod> <OPTIONS>
```
| Flag | Description | Default |
| ------------------------------------ | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| <nobr>`-namespace`, `-n`</nobr> | `String` The Kubernetes namespace to list proxies in. | Current [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) namespace. |
| <nobr>`-envoy-admin-endpoint`</nobr> | `String` Envoy sidecar address and port | `127.0.0.1:19000` |
#### Example Commands
The following example displays all transparent proxy upstreams in Consul service mesh from the given pod.
```shell-session
$ consul-k8s troubleshoot upstreams -pod frontend-767ccfc8f9-6f6gx
==> Upstreams (explicit upstreams only) (0)
==> Upstreams IPs (transparent proxy only) (1)
[10.4.6.160 240.0.0.3] true map[backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul]
If you cannot find the upstream address or cluster for a transparent proxy upstream:
- Check intentions: Tproxy upstreams are configured based on intentions. Make sure you have configured intentions to allow traffic to your upstream.
- To check that the right cluster is being dialed, run a DNS lookup for the upstream you are dialing. For example, run `dig backend.svc.consul` to return the IP address for the `backend` service. If the address you get from that is missing from the upstream IPs, it means that your proxy may be misconfigured.
```
The following example displays all explicit upstreams from the given pod in the Consul service mesh.
```shell-session
$ consul-k8s troubleshoot upstreams -pod client-767ccfc8f9-6f6gx
==> Upstreams (explicit upstreams only) (1)
server
counting
==> Upstreams IPs (transparent proxy only) (0)
If you cannot find the upstream address or cluster for a transparent proxy upstream:
- Check intentions: Tproxy upstreams are configured based on intentions. Make sure you have configured intentions to allow traffic to your upstream.
- To check that the right cluster is being dialed, run a DNS lookup for the upstream you are dialing. For example, run `dig backend.svc.consul` to return the IP address for the `backend` service. If the address you get from that is missing from the upstream IPs, it means that your proxy may be misconfigured.
```
### `troubleshoot proxy`
```shell-session
$ consul-k8s troubleshoot proxy -pod <pod> -upstream-ip <ip> <OPTIONS>
$ consul-k8s troubleshoot proxy -pod <pod> -upstream-envoy-id <envoy-id> <OPTIONS>
```
| Flag | Description | Default |
| ------------------------------------ | ----------------------------------------------------------| ---------------------------------------------------------------------------------------------------------------------- |
| <nobr>`-namespace`, `-n`</nobr> | `String` The Kubernetes namespace to list proxies in. | Current [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) namespace. |
| <nobr>`-envoy-admin-endpoint`</nobr> | `String` Envoy sidecar address and port | `127.0.0.1:19000` |
| <nobr>`-upstream-ip`</nobr> | `String` The IP address of the upstream transparent proxy | |
| <nobr>`-upstream-envoy-id`</nobr> | `String` The Envoy identifier of the upstream | |
#### Example Commands
The following example troubleshoots the Consul service mesh configuration and network issues between the given pod and the given upstream IP.
```shell-session
$ consul-k8s troubleshoot proxy -pod frontend-767ccfc8f9-6f6gx -upstream-ip 10.4.6.160
==> Validation
✓ certificates are valid
✓ Envoy has 0 rejected configurations
✓ Envoy has detected 0 connection failure(s)
✓ listener for upstream "backend" found
✓ route for upstream "backend" found
✓ cluster "backend.default.dc1.internal..consul" for upstream "backend" found
✓ healthy endpoints for cluster "backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
✓ cluster "backend2.default.dc1.internal..consul" for upstream "backend" found
! no healthy endpoints for cluster "backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
```
The following example troubleshoots the Consul service mesh configuration and network issues between the given pod and the given upstream.
```shell-session
$ consul-k8s troubleshoot proxy -pod frontend-767ccfc8f9-6f6gx -upstream-envoy-id db
==> Validation
✓ certificates are valid
✓ Envoy has 0 rejected configurations
✓ Envoy has detected 0 connection failure(s)
! no listener for upstream "db" found
! no route for upstream "backend" found
! no cluster "db.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "db" found
! no healthy endpoints for cluster "db.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "db" found
```
### `uninstall`
The `uninstall` command removes Consul from Kubernetes.

17
website/data/commands-nav-data.json

@ -528,6 +528,23 @@
}
]
},
{
"title": "troubleshoot",
"routes": [
{
"title": "Overview",
"path": "troubleshoot"
},
{
"title": "upstreams",
"path": "troubleshoot/upstreams"
},
{
"title": "proxy",
"path": "troubleshoot/proxy"
}
]
},
{
"title": "validate",
"path": "validate"

Loading…
Cancel
Save