From c9c34d0e76c2afdbb4dc5900dc324f17783e6c09 Mon Sep 17 00:00:00 2001 From: Daniel Upton Date: Mon, 10 Jan 2022 11:40:25 +0000 Subject: [PATCH 1/7] docs: fix placement of warning in kv put example --- website/content/commands/kv/put.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/content/commands/kv/put.mdx b/website/content/commands/kv/put.mdx index 76778d088e..0e42c106b7 100644 --- a/website/content/commands/kv/put.mdx +++ b/website/content/commands/kv/put.mdx @@ -69,6 +69,9 @@ $ consul kv put redis/config/connections Success! Data written to: redis/config/connections ``` +!> **Be careful when overwriting data!** The above operation would overwrite +the value at the key to the empty value. + If the `-base64` flag is set, the data will be decoded before writing: ```shell-session @@ -76,9 +79,6 @@ $ consul kv put -base64 foo/encoded aGVsbG8gd29ybGQK Success! Data written to: foo/encoded ``` -!> **Be careful when overwriting data!** The above operation would overwrite -the value at the key to the empty value. - For longer or sensitive values, it is possible to read from a file by prefixing with the `@` symbol: From 8630f031301fb8342e56a1adb6b21cd19e557224 Mon Sep 17 00:00:00 2001 From: Daniel Upton Date: Mon, 10 Jan 2022 12:15:59 +0000 Subject: [PATCH 2/7] docs: improve read/scanability of kv put examples - Split examples into sections with headers - Hide the clipboard on examples as the copied text isn't useful - Add an example of supplying data in a heredoc - Move the flags section to the bottom to clearly separate it from CAS which also mentions "flags" of a different kind - Slight re-wording for clarity --- website/content/commands/kv/put.mdx | 73 +++++++++++++++++++---------- 1 file changed, 48 insertions(+), 25 deletions(-) diff --git a/website/content/commands/kv/put.mdx b/website/content/commands/kv/put.mdx index 0e42c106b7..beb76fbf5b 100644 --- a/website/content/commands/kv/put.mdx +++ b/website/content/commands/kv/put.mdx @@ -57,57 +57,76 @@ Usage: `consul kv put [options] KEY [DATA]` To insert a value of "5" for the key named "redis/config/connections" in the KV store: -```shell-session +```shell-session hideClipboard $ consul kv put redis/config/connections 5 Success! Data written to: redis/config/connections ``` If no data is specified, the key will be created with empty data: -```shell-session +```shell-session hideClipboard $ consul kv put redis/config/connections Success! Data written to: redis/config/connections ``` -!> **Be careful when overwriting data!** The above operation would overwrite -the value at the key to the empty value. +!> **Be careful of overwriting data!** The above operation would overwrite +any existing value at the key to the empty value. -If the `-base64` flag is set, the data will be decoded before writing: +### Base64 Encoded Values -```shell-session +If the `-base64` flag is set, the given data will be Base64-decoded before writing: + +```shell-session hideClipboard $ consul kv put -base64 foo/encoded aGVsbG8gd29ybGQK Success! Data written to: foo/encoded ``` -For longer or sensitive values, it is possible to read from a file by prefixing -with the `@` symbol: +### Longer or Sensitive Values -```shell-session +For longer or sensitive values, it is possible to read from a file by +supplying its path prefixed with the `@` symbol: + +```shell-session hideClipboard $ consul kv put redis/config/password @password.txt -Success! Data written to: redis/config/connections +Success! Data written to: redis/config/password ``` Or read values from stdin by specifying the `-` symbol: -```shell-session -$ echo "5" | consul kv put redis/config/password - +```shell-session hideClipboard +$ echo "5" | consul kv put redis/config/connections - Success! Data written to: redis/config/connections +``` -$ consul kv put redis/config/password - +```shell-session hideClipboard +$ consul kv put redis/config/connections - 5 Success! Data written to: redis/config/connections ``` +```shell-session hideClipboard +$ consul kv put leaderboard/scores - < For secret and sensitive values, you should consider using a secret management solution like **[HashiCorp's Vault](https://www.vaultproject.io/)**. -While it is possible to secure values in Consul's KV store, Vault provides a -more robust interface for secret management. +While it is possible to encrpyt data before writing it to Consul's KV store, +Consul provides no built-in support for encryption at-rest. + +### Atomic Check-And-Set (CAS) To only update a key if it has not been modified since a given index, specify the `-cas` and `-modify-index` flags: -```shell-session +```shell-session hideClipboard $ consul kv get -detailed redis/config/connections | grep ModifyIndex ModifyIndex 456 @@ -118,24 +137,18 @@ $ consul kv put -cas -modify-index=456 redis/config/connections 10 Success! Data written to: redis/config/connections ``` -To specify flags on the key, use the `-flags` option. These flags are completely -controlled by the user: - -```shell-session -$ consul kv put -flags=42 redis/config/password s3cr3t -Success! Data written to: redis/config/password -``` +### Locking Primatives To create or tune a lock, use the `-acquire` and `-session` flags. The session must already exist (this command will not create it or manage it): -```shell-session +```shell-session hideClipboard $ consul kv put -acquire -session=abc123 redis/lock/update Success! Lock acquired on: redis/lock/update ``` When you are finished, release the lock: -```shell-session +```shell-session hideClipboard $ consul kv put -release -session=acb123 redis/lock/update Success! Lock released on: redis/lock/update ``` @@ -144,3 +157,13 @@ Success! Lock released on: redis/lock/update low-level primitives, you may want to look at the [consul lock](/commands/lock) command. It provides higher-level functionality without exposing the internal APIs of Consul. + +### Flags + +To set user-defined flags on the entry, use the `-flags` option. These flags +are completely controlled by the user and have no special meaning to Consul: + +```shell-session hideClipboard +$ consul kv put -flags=42 redis/config/password s3cr3t +Success! Data written to: redis/config/password +``` From ce55cb70b893021bf20066be01e9267a4672246c Mon Sep 17 00:00:00 2001 From: Daniel Upton Date: Mon, 10 Jan 2022 12:30:28 +0000 Subject: [PATCH 3/7] docs: call out `kv import` and the transaction API --- website/content/commands/kv/put.mdx | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/website/content/commands/kv/put.mdx b/website/content/commands/kv/put.mdx index beb76fbf5b..d407df721e 100644 --- a/website/content/commands/kv/put.mdx +++ b/website/content/commands/kv/put.mdx @@ -9,6 +9,11 @@ Command: `consul kv put` The `kv put` command writes the data to the given path in the KV store. +-> When writing multiple entries at once, consider using +[`kv import`](/commands/kv/import) instead. Alternatively, the +[transaction API](/api-docs/txn) provides support for performing up to +64 KV operations atomically. + ## Usage Usage: `consul kv put [options] KEY [DATA]` From 2dc05b4017e334b6bd7bd627d43d062dc52ba55e Mon Sep 17 00:00:00 2001 From: Daniel Upton Date: Mon, 10 Jan 2022 13:38:18 +0000 Subject: [PATCH 4/7] docs: improve kv get examples - Split examples into sections with headers - Hide the clipboard on examples as the copied text isn't useful - Format inline flags as code using backticks --- website/content/commands/kv/get.mdx | 52 +++++++++++++++++------------ 1 file changed, 30 insertions(+), 22 deletions(-) diff --git a/website/content/commands/kv/get.mdx b/website/content/commands/kv/get.mdx index c2f8bf913a..eb84020b69 100644 --- a/website/content/commands/kv/get.mdx +++ b/website/content/commands/kv/get.mdx @@ -53,17 +53,27 @@ Usage: `consul kv get [options] [KEY_OR_PREFIX]` To retrieve the value for the key named "redis/config/connections" in the KV store: -```shell-session +```shell-session hideClipboard $ consul kv get redis/config/connections 5 ``` -This will return the original, raw value stored in Consul. To view detailed -information about the key, specify the "-detailed" flag. This will output all -known metadata about the key including ModifyIndex and any user-supplied -flags: +This will return the original, raw value stored in Consul. -```shell-session +If the key with the given name does not exist, an error is returned: + +```shell-session hideClipboard +$ consul kv get not-a-real-key +Error! No key exists at: not-a-real-key +``` + +### Detailed Output + +To view detailed information about the key, specify the `-detailed` flag. +This will output all known metadata about the key including ModifyIndex +and any user-supplied flags: + +```shell-session hideClipboard $ consul kv get -detailed redis/config/connections CreateIndex 336 Flags 0 @@ -74,26 +84,22 @@ Session - Value 5 ``` -If the key with the given name does not exist, an error is returned: +### Recursively Reading By Prefix -```shell-session -$ consul kv get not-a-real-key -Error! No key exists at: not-a-real-key -``` +To treat the path as a prefix and list all entries which start with the given +prefix, specify the `-recurse` flag: -To treat the path as a prefix and list all keys which start with the given -prefix, specify the "-recurse" flag: - -```shell-session +```shell-session hideClipboard $ consul kv get -recurse redis/ redis/config/connections:5 redis/config/cpu:128 redis/config/memory:512 ``` -Or list detailed information about all pairs under a prefix: +Or combine with the `-detailed` flag to list detailed information about all +entries under a prefix: -```shell-session +```shell-session hideClipboard $ consul kv get -recurse -detailed redis CreateIndex 336 Flags 0 @@ -120,10 +126,12 @@ Session - Value 512 ``` -To just list the keys which start with the specified prefix, use the "-keys" +### Listing Keys + +To just list the keys which start with the specified prefix, use the `-keys` option instead. This is more performant and results in a smaller payload: -```shell-session +```shell-session hideClipboard $ consul kv get -keys redis/config/ redis/config/connections redis/config/cpu @@ -134,7 +142,7 @@ By default, the `-keys` operation uses a separator of "/", meaning it will not recurse beyond that separator. You can choose a different separator by setting `-separator=""`. -```shell-session +```shell-session hideClipboard $ consul kv get -keys -separator="c" redis redis/c ``` @@ -142,7 +150,7 @@ redis/c Alternatively, you can disable the separator altogether by setting it to the empty string: -```shell-session +```shell-session hideClipboard $ consul kv get -keys -separator="" redis redis/config/connections redis/config/cpu @@ -151,7 +159,7 @@ redis/config/memory To list all keys at the root, simply omit the prefix parameter: -```shell-session +```shell-session hideClipboard $ consul kv get -keys memcached/ redis/ From 021537c83743ff0b21788a9fc7320c3a953312a2 Mon Sep 17 00:00:00 2001 From: Daniel Upton Date: Mon, 10 Jan 2022 13:49:17 +0000 Subject: [PATCH 5/7] docs: call out `kv export` and the transaction API --- website/content/commands/kv/get.mdx | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/website/content/commands/kv/get.mdx b/website/content/commands/kv/get.mdx index eb84020b69..84f2e8b5e5 100644 --- a/website/content/commands/kv/get.mdx +++ b/website/content/commands/kv/get.mdx @@ -12,6 +12,12 @@ store at the given key name. If no key exists with that name, an error is returned. If a key exists with that name but has no data, nothing is returned. A key name or prefix is required. +-> When reading many entries under a given prefix, it may be worth considering +[`kv export`](/commands/kv/export) instead, the output of which can be used +with [`kv import`](/commands/kv/import) to move entire trees between Consul +clusters. Alternatively, the [transaction API](/api-docs/txn) provides +support for performing up to 64 KV operations atomically. + ## Usage Usage: `consul kv get [options] [KEY_OR_PREFIX]` From 8529a23e59bbfd14a281ec91a96ace5e0319b4fd Mon Sep 17 00:00:00 2001 From: Daniel Upton Date: Mon, 10 Jan 2022 13:59:43 +0000 Subject: [PATCH 6/7] docs: clarify transaction usage and limitations in kv api docs --- website/content/api-docs/kv.mdx | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/website/content/api-docs/kv.mdx b/website/content/api-docs/kv.mdx index cc99ed373d..05d0fca302 100644 --- a/website/content/api-docs/kv.mdx +++ b/website/content/api-docs/kv.mdx @@ -18,14 +18,16 @@ replication between datacenters, please view the ~> Values in the KV store cannot be larger than 512kb. -For multi-key updates, please consider using [transaction](/api/txn). +In order to perform atomic operations on multiple KV pairs (up to a limit of 64) +please consider using [transactions](/api/txn) instead. ## Read Key This endpoint returns the specified key. If no key exists at the given path, a 404 is returned instead of a 200 response. -For multi-key reads, please consider using [transaction](/api/txn). +For multi-key reads (up to a limit of 64) please consider using +[transactions](/api/txn) instead. | Method | Path | Produces | | ------ | ---------- | ------------------ | From 0e5c1c349c7b7500136c2d8fb7f4b90ad92a28f4 Mon Sep 17 00:00:00 2001 From: Daniel Upton Date: Mon, 10 Jan 2022 15:53:41 +0000 Subject: [PATCH 7/7] Incorporate feedback from @jkirschner-hashicorp and @karl-cardenas-coding --- website/content/api-docs/kv.mdx | 2 +- website/content/commands/kv/get.mdx | 18 +++++++++--------- website/content/commands/kv/put.mdx | 8 ++++---- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/website/content/api-docs/kv.mdx b/website/content/api-docs/kv.mdx index 05d0fca302..05525e8963 100644 --- a/website/content/api-docs/kv.mdx +++ b/website/content/api-docs/kv.mdx @@ -26,7 +26,7 @@ please consider using [transactions](/api/txn) instead. This endpoint returns the specified key. If no key exists at the given path, a 404 is returned instead of a 200 response. -For multi-key reads (up to a limit of 64) please consider using +For multi-key reads (up to a limit of 64 KV operations) please consider using [transactions](/api/txn) instead. | Method | Path | Produces | diff --git a/website/content/commands/kv/get.mdx b/website/content/commands/kv/get.mdx index 84f2e8b5e5..924ddf1a0f 100644 --- a/website/content/commands/kv/get.mdx +++ b/website/content/commands/kv/get.mdx @@ -12,10 +12,10 @@ store at the given key name. If no key exists with that name, an error is returned. If a key exists with that name but has no data, nothing is returned. A key name or prefix is required. --> When reading many entries under a given prefix, it may be worth considering -[`kv export`](/commands/kv/export) instead, the output of which can be used -with [`kv import`](/commands/kv/import) to move entire trees between Consul -clusters. Alternatively, the [transaction API](/api-docs/txn) provides +-> **Note**: When reading many entries under a given prefix, it may be worth +considering [`kv export`](/commands/kv/export) instead. The kv export output +can be used with [`kv import`](/commands/kv/import) to move entire trees between +Consul clusters. Alternatively, the [transaction API](/api-docs/txn) provides support for performing up to 64 KV operations atomically. ## Usage @@ -64,9 +64,9 @@ $ consul kv get redis/config/connections 5 ``` -This will return the original, raw value stored in Consul. +This will return the original raw value stored in Consul. -If the key with the given name does not exist, an error is returned: +If the key with the given name does not exist, an error is returned. ```shell-session hideClipboard $ consul kv get not-a-real-key @@ -76,7 +76,7 @@ Error! No key exists at: not-a-real-key ### Detailed Output To view detailed information about the key, specify the `-detailed` flag. -This will output all known metadata about the key including ModifyIndex +This will output all known metadata about the key including `ModifyIndex` and any user-supplied flags: ```shell-session hideClipboard @@ -102,8 +102,8 @@ redis/config/cpu:128 redis/config/memory:512 ``` -Or combine with the `-detailed` flag to list detailed information about all -entries under a prefix: +Alternatively, combine with the `-detailed` flag to list detailed information +about all entries under a prefix: ```shell-session hideClipboard $ consul kv get -recurse -detailed redis diff --git a/website/content/commands/kv/put.mdx b/website/content/commands/kv/put.mdx index d407df721e..0f89291768 100644 --- a/website/content/commands/kv/put.mdx +++ b/website/content/commands/kv/put.mdx @@ -9,7 +9,7 @@ Command: `consul kv put` The `kv put` command writes the data to the given path in the KV store. --> When writing multiple entries at once, consider using +-> **Note**: When writing multiple entries at once, consider using [`kv import`](/commands/kv/import) instead. Alternatively, the [transaction API](/api-docs/txn) provides support for performing up to 64 KV operations atomically. @@ -121,8 +121,8 @@ EOF Success! Data written to: leaderboard/scores ``` -~> For secret and sensitive values, you should consider using a secret -management solution like **[HashiCorp's Vault](https://www.vaultproject.io/)**. +~> **Warning**: For secret and sensitive values, you should consider using a +secret management solution like **[HashiCorp's Vault](https://learn.hashicorp.com/tutorials/vault/static-secrets?in=vault/secrets-management)**. While it is possible to encrpyt data before writing it to Consul's KV store, Consul provides no built-in support for encryption at-rest. @@ -142,7 +142,7 @@ $ consul kv put -cas -modify-index=456 redis/config/connections 10 Success! Data written to: redis/config/connections ``` -### Locking Primatives +### Locking Primitives To create or tune a lock, use the `-acquire` and `-session` flags. The session must already exist (this command will not create it or manage it):