github
/
consul
mirror of https://github.com/hashicorp/consul
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Custom Resource Definitions for Consul on k8s

pull/15384/head
boruszak 2022-11-15 11:10:48 -06:00
parent b0475b4f53
commit 92b5b4202a
1 changed files with 33 additions and 46 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

79
website/content/docs/k8s/crds/index.mdx
View File

@ -8,7 +8,7 @@ description: >-
# Custom Resource Definitions (CRDs) for Consul on Kubernetes # Custom Resource Definitions (CRDs) for Consul on Kubernetes
This topic describes how to manage Consul [configuration entries](/docs/agent/config-entries) This topic describes how to manage Consul [configuration entries](/docs/agent/config-entries)
via Kubernetes Custom Resources. Configuration entries provide cluster-wide defaults for the service mesh. with Kubernetes Custom Resources. Configuration entries provide cluster-wide defaults for the service mesh.
## Requirements ## Requirements
@ -50,8 +50,7 @@ Hang tight while we grab the latest from your chart repositories...
Update Complete. ⎈Happy Helming!⎈ Update Complete. ⎈Happy Helming!⎈
``` ```
Next, you must configure consul-helm via your `values.yaml` to install the custom resource definitions Next, you must configure consul-helm in your `values.yaml` file to install the custom resource definitions:
and enable the controller that acts on them:
<CodeBlockConfig filename="values.yaml" highlight="4-5,7-8"> <CodeBlockConfig filename="values.yaml" highlight="4-5,7-8">
@ -59,28 +58,21 @@ and enable the controller that acts on them:
global: global:
name: consul name: consul
controller:
enabled: true
connectInject: connectInject:
enabled: true enabled: true
``` ```
</CodeBlockConfig> </CodeBlockConfig>
Note that: Note that configuration entries are used to configure Consul service mesh, so `connectInject` must be enabled.
1. `controller.enabled: true` installs the CRDs and enables the controller. Refer to [Install with Helm Chart](/docs/k8s/installation/install) for further installation
1. Configuration entries are used to configure Consul service mesh so it's also
expected that `connectInject` will be enabled.
See [Install with Helm Chart](/docs/k8s/installation/install) for further installation
instructions. instructions.
## Upgrading An Existing Cluster to CRDs ## Upgrading An Existing Cluster to CRDs
If you have an existing Consul cluster running on Kubernetes you may need to perform If you have an existing Consul cluster running on Kubernetes you may need to perform
extra steps to migrate to CRDs. See [Upgrade An Existing Cluster to CRDs](/docs/k8s/crds/upgrade-to-crds) for full instructions. extra steps to migrate to CRDs. Refer to [Upgrade An Existing Cluster to CRDs](/docs/k8s/crds/upgrade-to-crds) for full instructions.
## Usage ## Usage
@ -88,7 +80,7 @@ Once installed, you can use `kubectl` to create and manage Consul's configuratio
### Create ### Create
You can create configuration entries via `kubectl apply`. You can create configuration entries with `kubectl apply`.
```shell-session ```shell-session
$ cat <<EOF | kubectl apply --filename - $ cat <<EOF | kubectl apply --filename -
@ -103,7 +95,7 @@ EOF
servicedefaults.consul.hashicorp.com/foo created servicedefaults.consul.hashicorp.com/foo created
``` ```
See [Configuration Entries](/docs/agent/config-entries) for detailed schema documentation. Refer to [Configuration Entries](/docs/agent/config-entries) for detailed schema documentation.
### Get ### Get
@ -121,7 +113,7 @@ in Consul.
### Describe ### Describe
You can use `kubectl describe [kind] [name]` to investigate the status of the You can use `kubectl describe [kind] [name]` to investigate the status of the
configuration entry. If `SYNCED` is false, the status will contain the reason configuration entry. If `SYNCED` is false, the status contains the reason
why. why.
```shell-session ```shell-session
@ -137,7 +129,7 @@ Status:
You can use `kubectl edit [kind] [name]` to edit the configuration entry: You can use `kubectl edit [kind] [name]` to edit the configuration entry:
```shell ```shell-session
$ kubectl edit servicedefaults foo $ kubectl edit servicedefaults foo
# change protocol: http => protocol: tcp # change protocol: http => protocol: tcp
servicedefaults.consul.hashicorp.com/foo edited servicedefaults.consul.hashicorp.com/foo edited
@ -171,11 +163,11 @@ Error from server (NotFound): servicedefaults.consul.hashicorp.com "foo" not fou
If running `kubectl delete` hangs without exiting, there may be If running `kubectl delete` hangs without exiting, there may be
a dependent configuration entry registered with Consul that prevents the target configuration entry from being a dependent configuration entry registered with Consul that prevents the target configuration entry from being
deleted. For example, if you set the protocol of your service to `http` via `ServiceDefaults` and then deleted. For example, if you set the protocol of your service to `http` in `ServiceDefaults` and then
create a `ServiceSplitter`, you won't be able to delete the `ServiceDefaults`. create a `ServiceSplitter`, you will not be able to delete `ServiceDefaults`.
This is because by deleting the `ServiceDefaults` config, you are setting the This is because by deleting the `ServiceDefaults` config, you are setting the
protocol back to the default which is `tcp`. Since `ServiceSplitter` requires protocol back to the default which is `tcp`. Because `ServiceSplitter` requires
that the service has an `http` protocol, Consul will not allow the `ServiceDefaults` that the service has an `http` protocol, Consul will not allow the `ServiceDefaults`
to be deleted since that would put Consul into a broken state. to be deleted since that would put Consul into a broken state.
@ -188,7 +180,7 @@ the `ServiceSplitter`.
Consul Open Source (Consul OSS) ignores Kubernetes namespaces and registers all services into the same Consul Open Source (Consul OSS) ignores Kubernetes namespaces and registers all services into the same
global Consul registry based on their names. For example, service `web` in Kubernetes namespace global Consul registry based on their names. For example, service `web` in Kubernetes namespace
`web-ns` and service `admin` in Kubernetes namespace `admin-ns` will be registered into `web-ns` and service `admin` in Kubernetes namespace `admin-ns` are registered into
Consul as `web` and `admin` with the Kubernetes source namespace ignored. Consul as `web` and `admin` with the Kubernetes source namespace ignored.
When creating custom resources to configure these services, the namespace of the When creating custom resources to configure these services, the namespace of the
@ -213,8 +205,7 @@ metadata:
spec: ... spec: ...
``` ```
~> **NOTE:** If two custom resources of the same kind **and** the same name are attempted to ~> **Note:** If you create two custom resources with identical `kind` and `name` values in different Kubernetes namespaces, the last one you create is not able to sync.
be created in different Kubernetes namespaces, the last one created will not be synced.
#### ServiceIntentions Special Case #### ServiceIntentions Special Case
@ -271,25 +262,24 @@ spec:
</CodeBlockConfig> </CodeBlockConfig>
~> **NOTE:** If two `ServiceIntentions` resources set the same `spec.destination.name`, the ~> **Note:** If two `ServiceIntentions` resources set the same `spec.destination.name`, the
last one created will not be synced. last one created is not synced.
### Consul Enterprise <EnterpriseAlert inline /> ### Consul Enterprise <EnterpriseAlert inline />
Consul Enterprise supports multiple configurations for how Kubernetes namespaces are mapped Consul Enterprise supports multiple configurations for how Kubernetes namespaces are mapped
to Consul namespaces. The Consul namespace that the custom resource is registered to Consul namespaces. The Consul namespace that the custom resource is registered
into depends on the configuration being used but in general, you should create your into depends on the configuration being used but in general, you should create your
custom resources in the same Kubernetes namespace as the service they're configuring and custom resources in the same Kubernetes namespace as the service they configure.
everything will work as expected.
The details on each configuration are: The details on each configuration are:
1. **Mirroring** - The Kubernetes namespace will be "mirrored" into Consul, i.e. 1. **Mirroring** - The Kubernetes namespace is "mirrored" into Consul. For example, the
service `web` in Kubernetes namespace `web-ns` will be registered as service `web` service `web` in Kubernetes namespace `web-ns` is registered as service `web`
in the Consul namespace `web-ns`. In the same vein, a `ServiceDefaults` custom resource with in the Consul namespace `web-ns`. In the same vein, a `ServiceDefaults` custom resource with
name `web` in Kubernetes namespace `web-ns` will configure that same service. name `web` in Kubernetes namespace `web-ns` configures that same service.
This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces): This is configured with [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
<CodeBlockConfig highlight="6-7"> <CodeBlockConfig highlight="6-7">
@ -305,13 +295,12 @@ The details on each configuration are:
</CodeBlockConfig> </CodeBlockConfig>
1. **Mirroring with prefix** - The Kubernetes namespace will be "mirrored" into Consul 1. **Mirroring with prefix** - The Kubernetes namespace is "mirrored" into Consul
with a prefix added to the Consul namespace, i.e. with a prefix added to the Consul namespace. For example, if the prefix is `k8s-` then service `web` in Kubernetes namespace `web-ns` will be registered as service `web`
if the prefix is `k8s-` then service `web` in Kubernetes namespace `web-ns` will be registered as service `web`
in the Consul namespace `k8s-web-ns`. In the same vein, a `ServiceDefaults` custom resource with in the Consul namespace `k8s-web-ns`. In the same vein, a `ServiceDefaults` custom resource with
name `web` in Kubernetes namespace `web-ns` will configure that same service. name `web` in Kubernetes namespace `web-ns` configures that same service.
This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces): This is configured with [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
<CodeBlockConfig highlight="8"> <CodeBlockConfig highlight="8">
@ -329,17 +318,16 @@ The details on each configuration are:
</CodeBlockConfig> </CodeBlockConfig>
1. **Single destination namespace** - The Kubernetes namespace is ignored and all services 1. **Single destination namespace** - The Kubernetes namespace is ignored and all services
will be registered into the same Consul namespace, i.e. if the destination Consul are registered into the same Consul namespace. For example, if the destination Consul
namespace is `my-ns` then service `web` in Kubernetes namespace `web-ns` will namespace is `my-ns` then service `web` in Kubernetes namespace `web-ns` is registered as service `web` in Consul namespace `my-ns`.
be registered as service `web` in Consul namespace `my-ns`.
In this configuration, the Kubernetes namespace of the custom resource is ignored. In this configuration, the Kubernetes namespace of the custom resource is ignored.
For example, a `ServiceDefaults` custom resource with the name `web` in Kubernetes For example, a `ServiceDefaults` custom resource with the name `web` in Kubernetes
namespace `admin-ns` will configure the service with name `web` even though that namespace `admin-ns` configures the service with name `web` even though that
service is running in Kubernetes namespace `web-ns` because the `ServiceDefaults` service is running in Kubernetes namespace `web-ns` because the `ServiceDefaults`
resource ends up registered into the same Consul namespace `my-ns`. resource ends up registered into the same Consul namespace `my-ns`.
This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces): This is configured with [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
<CodeBlockConfig highlight="7"> <CodeBlockConfig highlight="7">
@ -355,13 +343,12 @@ The details on each configuration are:
</CodeBlockConfig> </CodeBlockConfig>
~> **NOTE:** In this configuration, if two custom resources of the same kind **and** the same name are attempted to ~> **Note:** In this configuration, if two custom resources are created in two Kubernetes namespaces with identical `name` and `kind` values, the last one created is not synced.
be created in two Kubernetes namespaces, the last one created will not be synced.
#### ServiceIntentions Special Case (Enterprise) #### ServiceIntentions Special Case (Enterprise)
`ServiceIntentions` are different from the other custom resources because the `ServiceIntentions` are different from the other custom resources because the
name of the resource doesn't matter. For other resources, the name of the resource name of the resource does not matter. For other resources, the name of the resource
determines which service it configures. For example, this resource configures determines which service it configures. For example, this resource configures
the service `web`: the service `web`:
@ -379,7 +366,7 @@ spec:
</CodeBlockConfig> </CodeBlockConfig>
For `ServiceIntentions`, because we need to support the ability to create For `ServiceIntentions`, because we need to support the ability to create
wildcard intentions (e.g. `foo => * (allow)` meaning that `foo` can talk to **any** service), wildcard intentions (e.g. `foo => * (allow)` meaning that `foo` can talk to any service),
and because `*` is not a valid Kubernetes resource name, we instead use the field `spec.destination.name` and because `*` is not a valid Kubernetes resource name, we instead use the field `spec.destination.name`
to configure the destination service for the intention: to configure the destination service for the intention:
@ -415,5 +402,5 @@ spec:
In addition, we support the field `spec.destination.namespace` to configure In addition, we support the field `spec.destination.namespace` to configure
the destination service's Consul namespace. If `spec.destination.namespace` the destination service's Consul namespace. If `spec.destination.namespace`
is empty, then the Consul namespace used will be the same as the other is empty, then the Consul namespace used is the same as the other
config entries as outlined above. config entries as outlined above.