mirror of https://github.com/hashicorp/consul
438 lines
19 KiB
Markdown
438 lines
19 KiB
Markdown
---
|
|
layout: commands
|
|
page_title: 'Commands: Snapshot Agent'
|
|
description: |
|
|
The `consul snapshot agent` command starts a process that takes snapshots of the state of the Consul servers in Consul Enterprise. It can capture server state once or it can run as daemon that captures snapshots at defined intervals.
|
|
---
|
|
|
|
# Consul Snapshot Agent
|
|
|
|
Command: `consul snapshot agent`
|
|
|
|
<EnterpriseAlert />
|
|
|
|
~> The [`agent`](/consul/commands/snapshot/agent) subcommand described here is
|
|
available only in [Consul Enterprise](https://www.hashicorp.com/products/consul/)
|
|
version 0.7.1 and later. All other [snapshot subcommands](/consul/commands/snapshot)
|
|
are available in the open source version of Consul.
|
|
|
|
The `snapshot agent` subcommand starts a process that takes snapshots of the
|
|
state of the Consul servers and saves them locally, or pushes them to an
|
|
optional remote storage service.
|
|
|
|
The agent can be run as a long-running daemon process or in a one-shot mode
|
|
from a batch job, based on the [`-interval`](#interval) argument. Snapshotting
|
|
a remote datacenter is only available in one-shot mode.
|
|
|
|
As a long-running daemon, the agent will perform a leader election so multiple
|
|
processes can be run in a highly available fashion with automatic failover. The
|
|
agent will also register itself with Consul as a service, along with health
|
|
checks that show the agent is alive ("Consul Snapshot Agent Alive") and able to
|
|
take snapshots ("Consul Snapshot Agent Saving Snapshots"). The latter check is
|
|
only added on agents who have become a leader, so it's possible for operators to
|
|
tell which instances are alive and on standby and which instance has become
|
|
leader and starting saving snapshots.
|
|
|
|
As snapshots are saved, they will be reported in the log produced by the agent:
|
|
|
|
```log
|
|
2016/11/16 21:21:13 [INFO] Snapshot agent running
|
|
2016/11/16 21:21:13 [INFO] Waiting to obtain leadership...
|
|
2016/11/16 21:21:13 [INFO] Obtained leadership
|
|
2016/11/16 21:21:13 [INFO] Saved snapshot 1479360073448728784
|
|
```
|
|
|
|
The number shown with the saved snapshot is its ID, which is based on a UNIX
|
|
timestamp with nanosecond resolution, so collisions are unlikely and IDs are
|
|
monotonically increasing with time. This makes it easy to locate the latest
|
|
snapshot, even if the log data isn't available. The snapshot ID always appears
|
|
in the file name when using local storage, or in the object key when using
|
|
remote storage.
|
|
|
|
Snapshots can be restored using the
|
|
[`consul snapshot restore`](/consul/commands/snapshot/restore) command, or
|
|
the [HTTP API](/consul/api-docs/snapshot).
|
|
|
|
## ACL permissions
|
|
|
|
If ACLs are enabled the following privileges are required:
|
|
|
|
| Resource | Segment | Permission | Explanation |
|
|
| --------- | ---------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `acl` | N/A | `write` | All snapshotting operations require this privilege due to snapshots containing ACL tokens including unredacted secrets. |
|
|
| `key` | `<lock key>` | `write` | The lock key (which defaults to `consul-snapshot/lock`) is used during snapshot agent leader election. |
|
|
| `session` | `<agent name>` | `write` | The session used for locking during leader election is created against the agent name of the Consul agent that the Snapshot agent is registering itself with. |
|
|
| `service` | `<service name>` | `write` | The Snapshot agent registers itself with the local Consul agent and must have write privileges on its service name which is configured with `-service`. |
|
|
|
|
### Example ACL policy
|
|
|
|
The following is a example least privilege policy which allows the snapshot agent
|
|
to run on a node named `server-1234`.
|
|
|
|
<CodeTabs>
|
|
|
|
```hcl
|
|
# Required to read and snapshot ACL data
|
|
acl = "write"
|
|
# Allow the snapshot agent to create the key consul-snapshot/lock which will
|
|
# serve as a leader election lock when multiple snapshot agents are running in
|
|
# an environment
|
|
key "consul-snapshot/lock" {
|
|
policy = "write"
|
|
}
|
|
# Allow the snapshot agent to create sessions on the specified node
|
|
session "server-1234" {
|
|
policy = "write"
|
|
}
|
|
# Allow the snapshot agent to register itself into the catalog
|
|
service "consul-snapshot" {
|
|
policy = "write"
|
|
}
|
|
```
|
|
|
|
```json
|
|
{
|
|
"acl": "write",
|
|
"key": {
|
|
"consul-snapshot/lock": {
|
|
"policy": "write"
|
|
}
|
|
},
|
|
"session": {
|
|
"server-1234": {
|
|
"policy": "write"
|
|
}
|
|
},
|
|
"service": {
|
|
"consul-snapshot": {
|
|
"policy": "write"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
</CodeTabs>
|
|
|
|
Additional `session` rules should be created, or `session_prefix` used, if the
|
|
snapshot agent is deployed across more than one host.
|
|
|
|
## Usage
|
|
|
|
Usage: `consul snapshot agent [options]`
|
|
|
|
#### Config File Options
|
|
|
|
- `-config-dir` - Directory to look for JSON config files. Files will be read in
|
|
alphabetical order and must end with the extension ".json". This won't
|
|
recursively descend directories. This can be specified multiple times on the
|
|
command line.
|
|
|
|
- `-config-file` - File to read JSON configuration from. Files must end with the
|
|
extension ".json". This can be specified multiple times on the command line.
|
|
|
|
Config files referenced using `-config-dir` and `-config-file` have the following
|
|
format (shown populated with default values):
|
|
|
|
```json
|
|
{
|
|
"snapshot_agent": {
|
|
"http_addr": "127.0.0.1:8500",
|
|
"token": "",
|
|
"datacenter": "",
|
|
"ca_file": "",
|
|
"ca_path": "",
|
|
"cert_file": "",
|
|
"key_file": "",
|
|
"license_path": "",
|
|
"tls_server_name": "",
|
|
"login": {
|
|
"auth_method": "",
|
|
"bearer_token": "",
|
|
"bearer_token_file": "",
|
|
"meta": {},
|
|
},
|
|
"log": {
|
|
"level": "INFO",
|
|
"enable_syslog": false,
|
|
"syslog_facility": "LOCAL0"
|
|
},
|
|
"snapshot": {
|
|
"interval": "1h",
|
|
"retain": 30,
|
|
"stale": false,
|
|
"service": "consul-snapshot",
|
|
"deregister_after": "72h",
|
|
"lock_key": "consul-snapshot/lock",
|
|
"max_failures": 3,
|
|
"local_scratch_path": ""
|
|
},
|
|
"local_storage": {
|
|
"path": "."
|
|
},
|
|
"aws_storage": {
|
|
"access_key_id": "",
|
|
"secret_access_key": "",
|
|
"session_token": "",
|
|
"iam_endpoint": "",
|
|
"role_arn": "",
|
|
"role_session_name": "",
|
|
"web_identity_token_file": "",
|
|
"sts_endpoint": "",
|
|
"s3_region": "",
|
|
"s3_bucket": "",
|
|
"s3_key_prefix": "consul-snapshot",
|
|
"s3_server_side_encryption": false,
|
|
"s3_static_snapshot_name": "",
|
|
"s3_force_path_style": false
|
|
},
|
|
"azure_blob_storage": {
|
|
"account_name": "",
|
|
"account_key": "",
|
|
"container_name": ""
|
|
},
|
|
"google_storage": {
|
|
"bucket": ""
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
All fields are optional, and config files without a `snapshot_agent` object will
|
|
be ignored. At least one config file needs to have a `snapshot_agent` object, or the
|
|
snapshot agent will fail to start. The Consul agent is set up to ignore any
|
|
`snapshot_agent` object, so it's safe to use common config directories for both agents
|
|
if desired.
|
|
|
|
#### Snapshot Options
|
|
|
|
- `-interval` - Interval at which to perform snapshots as
|
|
a time with a unit suffix, which can be "s", "m", "h" for seconds, minutes, or
|
|
hours. If 0 is provided, the agent will take a single snapshot and then exit, which
|
|
is useful for running snapshots via batch jobs. Defaults to "1h"
|
|
|
|
- `-lock-key` - A prefix in Consul's KV store used to coordinate between
|
|
different instances of the snapshot agent order to only have one active instance
|
|
at a time. For highly available operation of the snapshot agent, simply run
|
|
multiple instances. All instances must be configured with the same lock key in
|
|
order to properly coordinate. Defaults to "consul-snapshot/lock".
|
|
|
|
- `-max-failures` - Number of snapshot failures after which the snapshot agent
|
|
will give up leadership. In a highly available operation with multiple snapshot
|
|
agents available, this gives another agent a chance to take over if an agent
|
|
is experiencing issues, such as running out of disk space for snapshots.
|
|
Defaults to 3.
|
|
|
|
- `-retain` - Number of snapshots to retain. After each snapshot is taken, the
|
|
oldest snapshots will start to be deleted in order to retain at most this many
|
|
snapshots. If this is set to 0, the agent will not perform this and snapshots
|
|
will accumulate forever. Defaults to 30.
|
|
|
|
- `-local-scratch-path` - Location to store all temporary snapshots in prior to
|
|
sending them off to the configured storage backend. If not configured the
|
|
system temporary directory will be used. When using the local storage backend
|
|
this is not configurable and `-local-path` will be used.
|
|
|
|
#### Agent Options
|
|
|
|
- `-deregister-after` - An interval, after which if the agent is unhealthy it will be
|
|
automatically deregistered from Consul service discovery. This is a time with a
|
|
unit suffix, which can be "s", "m", "h" for seconds, minutes, or hours. If 0 is
|
|
provided, this will be disabled. Defaults to "72h".
|
|
|
|
- `-log-level` - Controls verbosity of snapshot agent logs. Valid options are
|
|
"trace", "debug", "info", "warn", "error". Defaults to "info".
|
|
|
|
- `-service` - The service name to used when registering the agent with Consul.
|
|
Registering helps monitor running agents and the leader registers an additional
|
|
health check to monitor that snapshots are taking place. Defaults to
|
|
"consul-snapshot".
|
|
|
|
- `-syslog` - This enables forwarding logs to syslog. Defaults to false.
|
|
|
|
- `-syslog-facility` - Sets the facility to use for forwarding logs to syslog.
|
|
Defaults to "LOCAL0".
|
|
|
|
- `login-auth-method` - Auth method name to use to log into Consul. If provided, the token obtained with this auth method
|
|
will be used instead of a static token if it is provided. Currently, only `kubernetes` auth method type is supported.
|
|
|
|
- `login-bearer-token` - Bearer token to use to log into Consul. Used only if `-login-auth-method` is set.
|
|
|
|
- `login-bearer-token-file` - A file container bearer token to use for logging into Consul.
|
|
`-login-bearer-token` is ignored if this flag is provided.
|
|
|
|
- `login-meta` - Metadata to set on the token, formatted as key=value. This flag may be provided multiple times.
|
|
|
|
#### Local Storage Options
|
|
|
|
- `-local-path` - Location to store snapshots locally. The default behavior
|
|
of the snapshot agent is to store snapshots locally in this directory. Defaults
|
|
to "." to use the current working directory. If an alternate storage option is
|
|
configured, then local storage will be disabled and this option will be ignored.
|
|
|
|
#### S3 Storage Options
|
|
|
|
Note that despite the AWS references, any S3-compatible endpoint can be specified with `-aws-s3-endpoint`.
|
|
|
|
- `-aws-access-key-id`, `-aws-secret-access-key` and `-aws-session-token` - These arguments supply static
|
|
authentication information for connecting to S3. These may also be supplied using
|
|
the following alternative methods:<br />
|
|
|
|
- `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_SESSION_TOKEN` environment variables
|
|
- A credentials file (`~/.aws/credentials` or the file at the path specified by the
|
|
`AWS_SHARED_CREDENTIALS_FILE` environment variable)
|
|
- ECS task role metadata (container-specific)
|
|
- EC2 instance role metadata
|
|
|
|
- `-aws-iam-endpoint` - IAM endpoint to use when authenticating with static credentials.
|
|
Default is to use the global IAM endpoint.
|
|
|
|
- `-aws-role-arn`, `-aws-role-session-name`, and `-aws-web-identity-token-file` - These arguments are
|
|
used to obtain temporary credentials by assuming an IAM role. These may also be supplied using
|
|
the following alternative methods:<br />
|
|
|
|
- `AWS_ROLE_ARN`, `AWS_ROLE_SESSION_NAME`, and `AWS_WEB_IDENTITY_TOKEN_FILE` environment variables.
|
|
- A credentials file (~/.aws/credentials or the file at the path specified by the
|
|
`AWS_SHARED_CREDENTIALS_FILE` environment variable)
|
|
|
|
- `-aws-sts-endpoint` - STS endpoint to use for obtaining temporary credentials.
|
|
Default is to use the global STS endpoint.
|
|
|
|
- `-aws-s3-bucket` - S3 bucket to use. Required for S3 storage, and setting this
|
|
disables local storage. This should be only the bucket name without any
|
|
part of the key prefix.
|
|
|
|
- `-aws-s3-key-prefix` - Prefix to use for snapshot files in S3. Defaults to
|
|
"consul-snapshot".
|
|
|
|
- `-aws-s3-region` - S3 region to use. Required for S3 storage.
|
|
|
|
- `-aws-s3-endpoint` - S3 endpoint to use. Can also be specified using the
|
|
`AWS_S3_ENDPOINT` environment variable. Defaults to the regional S3 endpoint.
|
|
|
|
- `-aws-s3-server-side-encryption` - Enables saving snapshots to S3 using server side encryption with [Amazon S3-Managed Encryption Keys](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html)
|
|
|
|
- `-aws-s3-static-snapshot-name` - If this is given, all snapshots are saved with the same file name. The agent will not rotate or version snapshots, and will save them with the same name each time.
|
|
Use this if you want to rely on [S3's versioning capabilities](http://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html) instead of the agent handling it for you.
|
|
|
|
- `-aws-s3-force-path-style` - Enables the use of legacy path-based addressing instead of virtual addressing. This flag is required by minio
|
|
and other 3rd party S3 compatible object storage platforms where DNS or TLS requirements for virtual addressing are prohibitive.
|
|
For more information, refer to the AWS documentation on [Methods for accessing a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-bucket-intro.html)
|
|
|
|
- `-aws-s3-enable-kms` - Enables using [Amazon KMS](https://aws.amazon.com/kms/) for encrypting snapshots.
|
|
|
|
- `-aws-s3-kms-key` - Optional Amazon KMS key to use, if this is not set the default KMS master key will be used. Set this if you want to manage key rotation yourself.
|
|
|
|
-> When using a S3-compatible storage exposing a self-signed certificate the agent will not be able to perform
|
|
the snapshot operations unless the CA used to sign the storage certificate is trusted by the node running
|
|
the agent. You can add the CA root certificate to the OS trust store to have Consul trust the storage endpoint.
|
|
|
|
#### S3 Required Permissions
|
|
|
|
Different S3 permissions are required depending on the configuration of the snapshot agent. In particular extra permissions are required when
|
|
snapshot rotation is enabled. S3 storage snapshot rotation is enabled when the `retain` configuration is greater than 0 and when there is
|
|
no `aws-s3-static-snapshot-name` configured.
|
|
|
|
| Permission | Resource | When you need it |
|
|
| -------------------- | ---------------------------------- | ----------------------------------------------- |
|
|
| `PutObject` | `arn:aws:s3:::<bucket name>/<key>` | Required for all operations. |
|
|
| `DeleteObject` | `arn:aws:s3:::<bucket name>/<key>` | Required only when snapshot rotation is enabled |
|
|
| `ListBucket` | `arn:aws:s3:::<bucket name>` | Required only when snapshot rotation is enabled |
|
|
| `ListBucketVersions` | `arn:aws:s3:::<bucket name>` | Required only when snapshot rotation is enabled |
|
|
|
|
Within the table `<key>` refers to the the key used to store the snapshot. When `aws-s3-static-snapshot-name` is configured the `<key>` is simply the value of that configuration. Otherwise the `<key>` will be the `<aws-s3-key-prefix configuration>/consul-*.snap`.
|
|
|
|
The following example IAM policy document assumes that the `aws-s3-bucket` is `consul-data` with defaults for `aws-s3-key-prefix`, `aws-s3-static-snapshot-name` and `retain`:
|
|
|
|
```json
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [
|
|
{
|
|
"Sid": "",
|
|
"Effect": "Allow",
|
|
"Action": ["s3:PutObject", "s3:DeleteObject"],
|
|
"Resource": "arn:aws:s3:::consul-data/consul-snapshots/consul-*.snap"
|
|
},
|
|
{
|
|
"Sid": "",
|
|
"Effect": "Allow",
|
|
"Action": ["s3:ListBucketVersions", "s3:ListBucket"],
|
|
"Resource": "arn:aws:s3:::consul-data"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
#### Azure Blob Storage options
|
|
|
|
** Note: This currently only works on non-Solaris platforms due to library limitations **
|
|
|
|
From Consul Enterprise version `1.5.0` onwards, you can store snapshots in Azure Blob storage.
|
|
|
|
- `-azure-blob-account-name` and `-azure-blob-account-key` - These arguments supply
|
|
authentication information for connecting to Azure Blob storage.
|
|
|
|
- `-azure-blob-container-name` - Container to use. Required for Azure blob storage, and setting this
|
|
disables local storage.
|
|
|
|
* `-azure-blob-environment` - Environment to use. Defaults to AZUREPUBLICCLOUD. Other valid environments
|
|
are AZURECHINACLOUD, AZUREGERMANCLOUD and AZUREUSGOVERNMENTCLOUD. Introduced in Consul 1.7.3.
|
|
|
|
#### Google Cloud Storage options
|
|
|
|
From Consul Enterprise version `1.6.1` onwards, you can store snapshots in Google Cloud Storage. Authentication relies on automatic discovery through the sdk as described [here](https://cloud.google.com/docs/authentication/production):
|
|
|
|
- First, ADC checks to see if the environment variable GOOGLE_APPLICATION_CREDENTIALS is set. If the variable is set, ADC uses the service account file that the variable points to. The next section describes how to set the environment variable.
|
|
|
|
- If the environment variable isn't set, ADC uses the default service account that Compute Engine, Kubernetes Engine, App Engine, and Cloud Functions provide, for applications that run on those services.
|
|
|
|
- If ADC can't use either of the above credentials, an error occurs.
|
|
|
|
This integration needs the following information:
|
|
|
|
- `-gcs-bucket` supplies the bucket to use.
|
|
|
|
#### API Options
|
|
|
|
@include 'http_api_options_client.mdx'
|
|
|
|
## Examples
|
|
|
|
Running the agent with no arguments will run a long-running daemon process that will
|
|
perform leader election for highly available operation, register itself with Consul
|
|
service discovery with health checks, take snapshots every hour, retain the last 30
|
|
snapshots, and save snapshots into the current working directory:
|
|
|
|
```shell-session
|
|
$ consul snapshot agent
|
|
```
|
|
|
|
To run a one-shot backup, set the backup interval to 0. This will run a single snapshot
|
|
and delete any old snapshots based on the retain settings, but it will not perform any
|
|
leader election or service registration:
|
|
|
|
```shell-session
|
|
$ consul snapshot agent -interval=0
|
|
```
|
|
|
|
Please see the [HTTP API](/consul/api-docs/snapshot) documentation for
|
|
more details about snapshot internals.
|
|
|
|
## Licensing
|
|
|
|
The snapshot agent requires a license when it starts before it will perform any other
|
|
actions. This can be provided using the `license_path` configuration item, the
|
|
`CONSUL_LICENSE_PATH` environment variable or the `CONSUL_LICENSE` environment variable.
|
|
The `license_path` configuration and `CONSUL_LICENSE_PATH` variable should point to
|
|
files that contain the license whereas the `CONSUL_LICENSE` variable value should be
|
|
the contents of the license itself. If a license is present in multiple ways the
|
|
then the order of precedence is as follows:
|
|
|
|
1. `CONSUL_LICENSE` variable
|
|
2. `CONSUL_LICENSE_PATH` variable
|
|
3. `license_path` configuration.
|
|
|
|
The ability to load licenses from the configuration or environment was added in v1.10.0,
|
|
v1.9.7 and v1.8.13. See the [licensing documentation](/consul/docs/enterprise/license/overview) for
|
|
more information about Consul Enterprise license management.
|