mirror of https://github.com/hashicorp/consul
Custom Resource Definitions for Consul on k8s
boruszak
parent
b0475b4f53
commit
92b5b4202a
|
|
@ -8,7 +8,7 @@ description: >-
|
|||
# Custom Resource Definitions (CRDs) for Consul on Kubernetes
|
||||
|
||||
This topic describes how to manage Consul [configuration entries](/docs/agent/config-entries)
|
||||
via Kubernetes Custom Resources. Configuration entries provide cluster-wide defaults for the service mesh.
|
||||
with Kubernetes Custom Resources. Configuration entries provide cluster-wide defaults for the service mesh.
|
||||
|
||||
## Requirements
|
||||
|
||||
|
|
@ -50,8 +50,7 @@ Hang tight while we grab the latest from your chart repositories...
|
|||
Update Complete. ⎈Happy Helming!⎈
|
||||
```
|
||||
|
||||
Next, you must configure consul-helm via your `values.yaml` to install the custom resource definitions
|
||||
and enable the controller that acts on them:
|
||||
Next, you must configure consul-helm in your `values.yaml` file to install the custom resource definitions:
|
||||
|
||||
<CodeBlockConfig filename="values.yaml" highlight="4-5,7-8">
|
||||
|
||||
|
|
@ -59,28 +58,21 @@ and enable the controller that acts on them:
|
|||
global:
|
||||
name: consul
|
||||
|
||||
controller:
|
||||
enabled: true
|
||||
|
||||
connectInject:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
|
||||
Note that:
|
||||
Note that configuration entries are used to configure Consul service mesh, so `connectInject` must be enabled.
|
||||
|
||||
1. `controller.enabled: true` installs the CRDs and enables the controller.
|
||||
1. Configuration entries are used to configure Consul service mesh so it's also
|
||||
expected that `connectInject` will be enabled.
|
||||
|
||||
See [Install with Helm Chart](/docs/k8s/installation/install) for further installation
|
||||
Refer to [Install with Helm Chart](/docs/k8s/installation/install) for further installation
|
||||
instructions.
|
||||
|
||||
## Upgrading An Existing Cluster to CRDs
|
||||
|
||||
If you have an existing Consul cluster running on Kubernetes you may need to perform
|
||||
extra steps to migrate to CRDs. See [Upgrade An Existing Cluster to CRDs](/docs/k8s/crds/upgrade-to-crds) for full instructions.
|
||||
extra steps to migrate to CRDs. Refer to [Upgrade An Existing Cluster to CRDs](/docs/k8s/crds/upgrade-to-crds) for full instructions.
|
||||
|
||||
## Usage
|
||||
|
||||
|
|
@ -88,7 +80,7 @@ Once installed, you can use `kubectl` to create and manage Consul's configuratio
|
|||
|
||||
### Create
|
||||
|
||||
You can create configuration entries via `kubectl apply`.
|
||||
You can create configuration entries with `kubectl apply`.
|
||||
|
||||
```shell-session
|
||||
$ cat <<EOF | kubectl apply --filename -
|
||||
|
|
@ -103,7 +95,7 @@ EOF
|
|||
servicedefaults.consul.hashicorp.com/foo created
|
||||
```
|
||||
|
||||
See [Configuration Entries](/docs/agent/config-entries) for detailed schema documentation.
|
||||
Refer to [Configuration Entries](/docs/agent/config-entries) for detailed schema documentation.
|
||||
|
||||
### Get
|
||||
|
||||
|
|
@ -121,7 +113,7 @@ in Consul.
|
|||
### Describe
|
||||
|
||||
You can use `kubectl describe [kind] [name]` to investigate the status of the
|
||||
configuration entry. If `SYNCED` is false, the status will contain the reason
|
||||
configuration entry. If `SYNCED` is false, the status contains the reason
|
||||
why.
|
||||
|
||||
```shell-session
|
||||
|
|
@ -137,7 +129,7 @@ Status:
|
|||
|
||||
You can use `kubectl edit [kind] [name]` to edit the configuration entry:
|
||||
|
||||
```shell
|
||||
```shell-session
|
||||
$ kubectl edit servicedefaults foo
|
||||
# change protocol: http => protocol: tcp
|
||||
servicedefaults.consul.hashicorp.com/foo edited
|
||||
|
|
@ -171,11 +163,11 @@ Error from server (NotFound): servicedefaults.consul.hashicorp.com "foo" not fou
|
|||
|
||||
If running `kubectl delete` hangs without exiting, there may be
|
||||
a dependent configuration entry registered with Consul that prevents the target configuration entry from being
|
||||
deleted. For example, if you set the protocol of your service to `http` via `ServiceDefaults` and then
|
||||
create a `ServiceSplitter`, you won't be able to delete the `ServiceDefaults`.
|
||||
deleted. For example, if you set the protocol of your service to `http` in `ServiceDefaults` and then
|
||||
create a `ServiceSplitter`, you will not be able to delete `ServiceDefaults`.
|
||||
|
||||
This is because by deleting the `ServiceDefaults` config, you are setting the
|
||||
protocol back to the default which is `tcp`. Since `ServiceSplitter` requires
|
||||
protocol back to the default which is `tcp`. Because `ServiceSplitter` requires
|
||||
that the service has an `http` protocol, Consul will not allow the `ServiceDefaults`
|
||||
to be deleted since that would put Consul into a broken state.
|
||||
|
||||
|
|
@ -188,7 +180,7 @@ the `ServiceSplitter`.
|
|||
|
||||
Consul Open Source (Consul OSS) ignores Kubernetes namespaces and registers all services into the same
|
||||
global Consul registry based on their names. For example, service `web` in Kubernetes namespace
|
||||
`web-ns` and service `admin` in Kubernetes namespace `admin-ns` will be registered into
|
||||
`web-ns` and service `admin` in Kubernetes namespace `admin-ns` are registered into
|
||||
Consul as `web` and `admin` with the Kubernetes source namespace ignored.
|
||||
|
||||
When creating custom resources to configure these services, the namespace of the
|
||||
|
|
@ -213,8 +205,7 @@ metadata:
|
|||
spec: ...
|
||||
```
|
||||
|
||||
~> **NOTE:** If two custom resources of the same kind **and** the same name are attempted to
|
||||
be created in different Kubernetes namespaces, the last one created will not be synced.
|
||||
~> **Note:** If you create two custom resources with identical `kind` and `name` values in different Kubernetes namespaces, the last one you create is not able to sync.
|
||||
|
||||
#### ServiceIntentions Special Case
|
||||
|
||||
|
|
@ -271,25 +262,24 @@ spec:
|
|||
|
||||
</CodeBlockConfig>
|
||||
|
||||
~> **NOTE:** If two `ServiceIntentions` resources set the same `spec.destination.name`, the
|
||||
last one created will not be synced.
|
||||
~> **Note:** If two `ServiceIntentions` resources set the same `spec.destination.name`, the
|
||||
last one created is not synced.
|
||||
|
||||
### Consul Enterprise <EnterpriseAlert inline />
|
||||
|
||||
Consul Enterprise supports multiple configurations for how Kubernetes namespaces are mapped
|
||||
to Consul namespaces. The Consul namespace that the custom resource is registered
|
||||
into depends on the configuration being used but in general, you should create your
|
||||
custom resources in the same Kubernetes namespace as the service they're configuring and
|
||||
everything will work as expected.
|
||||
custom resources in the same Kubernetes namespace as the service they configure.
|
||||
|
||||
The details on each configuration are:
|
||||
|
||||
1. **Mirroring** - The Kubernetes namespace will be "mirrored" into Consul, i.e.
|
||||
service `web` in Kubernetes namespace `web-ns` will be registered as service `web`
|
||||
1. **Mirroring** - The Kubernetes namespace is "mirrored" into Consul. For example, the
|
||||
service `web` in Kubernetes namespace `web-ns` is registered as service `web`
|
||||
in the Consul namespace `web-ns`. In the same vein, a `ServiceDefaults` custom resource with
|
||||
name `web` in Kubernetes namespace `web-ns` will configure that same service.
|
||||
name `web` in Kubernetes namespace `web-ns` configures that same service.
|
||||
|
||||
This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
|
||||
This is configured with [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
|
||||
|
||||
<CodeBlockConfig highlight="6-7">
|
||||
|
||||
|
|
@ -305,13 +295,12 @@ The details on each configuration are:
|
|||
|
||||
</CodeBlockConfig>
|
||||
|
||||
1. **Mirroring with prefix** - The Kubernetes namespace will be "mirrored" into Consul
|
||||
with a prefix added to the Consul namespace, i.e.
|
||||
if the prefix is `k8s-` then service `web` in Kubernetes namespace `web-ns` will be registered as service `web`
|
||||
1. **Mirroring with prefix** - The Kubernetes namespace is "mirrored" into Consul
|
||||
with a prefix added to the Consul namespace. For example, if the prefix is `k8s-` then service `web` in Kubernetes namespace `web-ns` will be registered as service `web`
|
||||
in the Consul namespace `k8s-web-ns`. In the same vein, a `ServiceDefaults` custom resource with
|
||||
name `web` in Kubernetes namespace `web-ns` will configure that same service.
|
||||
name `web` in Kubernetes namespace `web-ns` configures that same service.
|
||||
|
||||
This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
|
||||
This is configured with [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
|
||||
|
||||
<CodeBlockConfig highlight="8">
|
||||
|
||||
|
|
@ -329,17 +318,16 @@ The details on each configuration are:
|
|||
</CodeBlockConfig>
|
||||
|
||||
1. **Single destination namespace** - The Kubernetes namespace is ignored and all services
|
||||
will be registered into the same Consul namespace, i.e. if the destination Consul
|
||||
namespace is `my-ns` then service `web` in Kubernetes namespace `web-ns` will
|
||||
be registered as service `web` in Consul namespace `my-ns`.
|
||||
are registered into the same Consul namespace. For example, if the destination Consul
|
||||
namespace is `my-ns` then service `web` in Kubernetes namespace `web-ns` is registered as service `web` in Consul namespace `my-ns`.
|
||||
|
||||
In this configuration, the Kubernetes namespace of the custom resource is ignored.
|
||||
For example, a `ServiceDefaults` custom resource with the name `web` in Kubernetes
|
||||
namespace `admin-ns` will configure the service with name `web` even though that
|
||||
namespace `admin-ns` configures the service with name `web` even though that
|
||||
service is running in Kubernetes namespace `web-ns` because the `ServiceDefaults`
|
||||
resource ends up registered into the same Consul namespace `my-ns`.
|
||||
|
||||
This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
|
||||
This is configured with [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
|
||||
|
||||
<CodeBlockConfig highlight="7">
|
||||
|
||||
|
|
@ -355,13 +343,12 @@ The details on each configuration are:
|
|||
|
||||
</CodeBlockConfig>
|
||||
|
||||
~> **NOTE:** In this configuration, if two custom resources of the same kind **and** the same name are attempted to
|
||||
be created in two Kubernetes namespaces, the last one created will not be synced.
|
||||
~> **Note:** In this configuration, if two custom resources are created in two Kubernetes namespaces with identical `name` and `kind` values, the last one created is not synced.
|
||||
|
||||
#### ServiceIntentions Special Case (Enterprise)
|
||||
|
||||
`ServiceIntentions` are different from the other custom resources because the
|
||||
name of the resource doesn't matter. For other resources, the name of the resource
|
||||
name of the resource does not matter. For other resources, the name of the resource
|
||||
determines which service it configures. For example, this resource configures
|
||||
the service `web`:
|
||||
|
||||
|
|
@ -379,7 +366,7 @@ spec:
|
|||
</CodeBlockConfig>
|
||||
|
||||
For `ServiceIntentions`, because we need to support the ability to create
|
||||
wildcard intentions (e.g. `foo => * (allow)` meaning that `foo` can talk to **any** service),
|
||||
wildcard intentions (e.g. `foo => * (allow)` meaning that `foo` can talk to any service),
|
||||
and because `*` is not a valid Kubernetes resource name, we instead use the field `spec.destination.name`
|
||||
to configure the destination service for the intention:
|
||||
|
||||
|
|
@ -415,5 +402,5 @@ spec:
|
|||
|
||||
In addition, we support the field `spec.destination.namespace` to configure
|
||||
the destination service's Consul namespace. If `spec.destination.namespace`
|
||||
is empty, then the Consul namespace used will be the same as the other
|
||||
is empty, then the Consul namespace used is the same as the other
|
||||
config entries as outlined above.
|
||||
|
|
|
|||
Loading…
Reference in New Issue