[NET-1151 NET-11046] docs: Add request normalization, L7 headers options, and security guidance to release/1.15.x (#21858)

backport of commit a9d70fef6980d492b774c58114dc72e7454bddc4 and 9e7757da16
pull/21889/head
Michael Zalimeni 2024-10-28 13:57:47 -04:00 committed by GitHub
parent 400c6a8bdc
commit 5cb55fc272
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 333 additions and 126 deletions

View File

@ -264,6 +264,60 @@ spec:
Note that the Kubernetes example does not include a `partition` field. Configuration entries are applied on Kubernetes using [custom resource definitions (CRD)](/consul/docs/k8s/crds), which can only be scoped to their own partition.
### Request Normalization
Enable options under `HTTP.Incoming.RequestNormalization` to apply normalization to all inbound traffic to mesh proxies.
~> **Compatibility warning**: This feature is available as of Consul CE 1.20.1 and Consul Enterprise 1.20.1, 1.19.2, 1.18.3, and 1.15.15. We recommend upgrading to the latest version of Consul to take advantage of the latest features and improvements.
<CodeTabs tabs={[ "HCL", "Kubernetes YAML", "JSON" ]}>
```hcl
Kind = "mesh"
HTTP {
Incoming {
RequestNormalization {
InsecureDisablePathNormalization = false // default false, shown for completeness
MergeSlashes = true
PathWithEscapedSlashesAction = "UNESCAPE_AND_FORWARD"
HeadersWithUnderscoresAction = "REJECT_REQUEST"
}
}
}
```
```yaml
apiVersion: consul.hashicorp.com/v1alpha1
kind: Mesh
metadata:
name: mesh
spec:
http:
incoming:
requestNormalization:
insecureDisablePathNormalization: false # default false, shown for completeness
mergeSlashes: true
pathWithEscapedSlashesAction: UNESCAPE_AND_FORWARD
headersWithUnderscoresAction: REJECT_REQUEST
```
```json
{
"Kind": "mesh",
"HTTP": {
"Incoming": {
"RequestNormalization": {
"InsecureDisablePathNormalization": false,
"MergeSlashes": true,
"PathWithEscapedSlashesAction": "UNESCAPE_AND_FORWARD",
"HeadersWithUnderscoresAction": "REJECT_REQUEST"
}
}
}
}
```
</CodeTabs>
## Available Fields
@ -435,6 +489,57 @@ Note that the Kubernetes example does not include a `partition` field. Configura
for all Envoy proxies. As a result, Consul will not include the \`x-forwarded-client-cert\` header in the next hop.
If set to \`false\` (default), the XFCC header is propagated to upstream applications.`,
},
{
name: 'Incoming',
type: 'DirectionalHTTPConfig: <optional>',
description: `HTTP configuration for inbound traffic to mesh proxies.`,
children: [
{
name: 'RequestNormalization',
type: 'RequestNormalizationConfig: <optional>',
description: `Request normalization configuration for inbound traffic to mesh proxies.`,
children: [
{
name: 'InsecureDisablePathNormalization',
type: 'bool: false',
description: `Sets the value of the \`normalize_path\` option in the Envoy listener's \`HttpConnectionManager\`. The default value is \`false\`.
When set to \`true\` in Consul, \`normalize_path\` is set to \`false\` for the Envoy proxy.
This parameter disables the normalization of request URL paths according to RFC 3986,
conversion of \`\\\` to \`/\`, and decoding non-reserved %-encoded characters. When using L7
intentions with path match rules, we recommend enabling path normalization in order
to avoid match rule circumvention with non-normalized path values.`,
},
{
name: 'MergeSlashes',
type: 'bool: false',
description: `Sets the value of the \`merge_slashes\` option in the Envoy listener's \`HttpConnectionManager\`. The default value is \`false\`.
This option controls the normalization of request URL paths by merging consecutive \`/\` characters. This normalization is not part
of RFC 3986. When using L7 intentions with path match rules, we recommend enabling this setting to avoid match rule circumvention through non-normalized path values, unless legitimate service
traffic depends on allowing for repeat \`/\` characters, or upstream services are configured to
differentiate between single and multiple slashes.`,
},
{
name: 'PathWithEscapedSlashesAction',
type: 'string: ""',
description: `Sets the value of the \`path_with_escaped_slashes_action\` option in the Envoy listener's
\`HttpConnectionManager\`. The default value of this option is empty, which is
equivalent to \`IMPLEMENTATION_SPECIFIC_DEFAULT\`. This parameter controls the action taken in response to request URL paths with escaped
slashes in the path. When using L7 intentions with path match rules, we recommend enabling this setting to avoid match rule circumvention through non-normalized path values, unless legitimate service
traffic depends on allowing for escaped \`/\` or \`\\\` characters, or upstream services are configured to
differentiate between escaped and unescaped slashes. Refer to the Envoy documentation for more information on available
options.`,
},
{
name: 'HeadersWithUnderscoresAction',
type: 'string: ""',
description: `Sets the value of the \`headers_with_underscores_action\` option in the Envoy listener's
\`HttpConnectionManager\` under \`common_http_protocol_options\`. The default value of this option is
empty, which is equivalent to \`ALLOW\`. Refer to the Envoy documentation for more information on available options.`,
},
],
},
],
}
],
},
{

View File

@ -41,7 +41,9 @@ The following outline shows how to format the service intentions configuration e
- [`Exact`](#sources-permissions-http-header): string | no default
- [`Prefix`](#sources-permissions-http-header): string | no default
- [`Suffix`](#sources-permissions-http-header): string | no default
- [`Regex`](#sources-permissions-http-header): string | no default
- [`Contains`](#spec-sources-permissions-http-header): string | no default
- [`Regex`](#spec-sources-permissions-http-header): string | no default
- [`IgnoreCase`](#spec-sources-permissions-http-header): boolean | `false`
- [`Invert`](#sources-permissions-http-header): boolean | `false`
- [`Precedence`](#sources-precedence): number | no default | _read-only_
- [`Type`](#sources-type): string | `consul`
@ -82,7 +84,9 @@ The following outline shows how to format the service intentions configuration e
- [`exact`](#spec-sources-permissions-http-header): string | no default
- [`prefix`](#spec-sources-permissions-http-header): string | no default
- [`suffix`](#spec-sources-permissions-http-header): string | no default
- [`contains`](#spec-sources-permissions-http-header): string | no default
- [`regex`](#spec-sources-permissions-http-header): string | no default
- [`ignoreCase`](#spec-sources-permissions-http-header): boolean | `false`
- [`invert`](#spec-sources-permissions-http-header): boolean | `false`
- [`type`](#spec-sources-type): string | `consul`
- [`description`](#spec-sources-description): string
@ -116,7 +120,7 @@ Sources = [
Action = "allow" or "deny" # string for L4 intentions
Permissions = [
{
Action = "allow" or "deny" # string for L7 intenions
Action = "allow" or "deny" # string for L7 intentions
HTTP = {
PathExact = "<exact path to match>" # string
PathPrefix = "<path prefix to match>" # string
@ -129,18 +133,31 @@ Sources = [
{
Name = "<http header name>" # string
Present = <true or false> # boolean
Invert = <true or false> # boolean
},
{
Name = "<http header name>" # string
Exact = "<header-value>" # boolean
IgnoreCase = <true or false> # boolean
Invert = <true or false> # boolean
},
{
Name = "<http header name>" # string
Prefix = "<source header value prefix>" # string
IgnoreCase = <true or false> # boolean
Invert = <true or false> # boolean
},
{
Name = "<http header name>" # string
Suffix = "<source header value suffix>" # string
IgnoreCase = <true or false> # boolean
Invert = <true or false> # boolean
},
{
Name = "<http header name>" # string
Contains = "<value to search for>" # string
IgnoreCase = <true or false> # boolean
Invert = <true or false> # boolean
},
{
Name = "<http header name>" # string
@ -194,12 +211,23 @@ spec:
header:
- name: <http header name>
present: true
invert: false
- name: <http header name>
exact: false
exact: <header-value>
ignoreCase: false
invert: false
- name: <http header name>
prefix: <source header value prefix>
ignoreCase: false
invert: false
- name: <http header name>
suffix: <source header value suffix>
ignoreCase: false
invert: false
- name: <http header name>
contains: <value to search for>
ignoreCase: false
invert: false
- name: <http header name>
regex: <regex pattern to match>
invert: false
@ -241,19 +269,32 @@ spec:
"Header":[
{
"Name":"<http header name>",
"Present":true
"Present":true,
"Invert":false
},
{
"Name":"<http header name>",
"Exact":false
"Exact":"<header-value>",
"IgnoreCase":false,,
"Invert":false
},
{
"Name":"<http header name>",
"Prefix":"<source header value prefix>"
"Prefix":"<source header value prefix>",
"IgnoreCase":false,
"Invert":false
},
{
"Name":"<http header name>",
"Suffix":"<source header value suffix>"
"Suffix":"<source header value suffix>",
"IgnoreCase":false,
"Invert":false
},
{
"Name":"<http header name>",
"Contains":"<value to search for>",
"IgnoreCase":false,
"Invert":false
},
{
"Name":"<http header name>",
@ -495,7 +536,9 @@ Each member of the `Header` list is a map that contains a `Name` field and at le
| `Exact` | Specifies a value for the header key set in the `Name` field. If the request header value matches the `Exact` value, Consul applies the permission. Do not specify `Exact` if `Present`, `Prefix`, `Suffix`, or `Regex` are configured in the same `Header` configuration. | string | optional |
| `Prefix` | Specifies a prefix value for the header key set in the `Name` field. If the request header value starts with the `Prefix` value, Consul applies the permission. Do not specify `Prefix` if `Present`, `Exact`, `Suffix`, or `Regex` are configured in the same `Header` configuration. | string | optional |
| `Suffix` | Specifies a suffix value for the header key set in the `Name` field. If the request header value ends with the `Suffix` value, Consul applies the permission. Do not specify `Suffix` if `Present`, `Exact`, `Prefix`, or `Regex` are configured in the same `Header` configuration. | string | optional |
| `Regex` | Specifies a regular expression pattern as the value for the header key set in the `Name` field. If the request header value matches the regex, Consul applies the permission. Do not specify `Regex` if `Present`, `Exact`, `Prefix`, or `Suffix` are configured in the same `Header` configuration. The regex syntax is proxy-specific. If using Envoy, refer to the [re2 documentation](https://github.com/google/re2/wiki/Syntax) for details. | string | optional |
| `Contains` | Specifies a contains value for the header key set in the `Name` field. If the request header value includes the `Contains` value, Consul applies the permission. Do not specify `Contains` if `Present`, `Exact`, `Prefix`, `Suffix`, or `Regex` are configured in the same `header` configuration. | string | optional |
| `Regex` | Specifies a regular expression pattern as the value for the header key set in the `Name` field. If the request header value matches the regex, Consul applies the permission. Do not specify `Regex` if `Present`, `Exact`, `Prefix`, `Suffix`, or `Contains` are configured in the same `Header` configuration. The regex syntax is proxy-specific. If using Envoy, refer to the [re2 documentation](https://github.com/google/re2/wiki/Syntax) for details. | string | optional |
| `IgnoreCase` | Ignores the case of the provided header value when matching with `Exact`, `Prefix`, `Suffix`, or `Contains`. Default is `false`. | boolean | optional |
| `Invert` | Inverts the matching logic configured in the `Header`. Default is `false`. | boolean | optional |
@ -753,16 +796,22 @@ Specifies a set of criteria for matching HTTP request headers. The request heade
- Default: None
- Data type: List of maps
Each member of the `header` list is a map that contains a `name` field and at least one match criterion. The following table describes the parameters that each member of the `header` list may contain:
Each member of the `header` list is a map that contains a `name` field and at least one match criterion.
~> **Warning**: If it is possible for a header to contain multiple values, we recommend using `contains` or `regex` rather than `exact`, `prefix`, or `suffix`. Envoy internally concatenates multiple header values into a single CSV value prior to applying match rules, which may result in match rules that depend on the beginning or end of a string vulnerable to circumvention. A more robust alternative is using `contains` or, if a stricter value match is required, configuring a regex pattern that is tolerant of comma-separated values. These options are available as of Consul CE 1.20.1 and Consul Enterprise 1.20.1, 1.19.2, 1.18.3, and 1.15.15.
The following table describes the parameters that each member of the `header` list may contain:
| Parameter | Description | Data type | Required |
| --- | --- | --- | --- |
| `name` | Specifies the name of the header to match. | string | required |
| `present` | Enables a match if the header configured in the `name` field appears in the request. Consul matches on any value as long as the header key appears in the request. Do not specify `present` if `exact`, `prefix`, `suffix`, or `regex` are configured in the same `header` configuration. | boolean | optional |
| `Exact` | Specifies a value for the header key set in the `Name` field. If the request header value matches the `exact` value, Consul applies the permission. Do not specify `exact` if `present`, `prefix`, `suffix`, or `regex` are configured in the same `header` configuration. | string | optional |
| `prefix` | Specifies a prefix value for the header key set in the `name` field. If the request header value starts with the `prefix` value, Consul applies the permission. Do not specify `prefix` if `present`, `exact`, `suffix`, or `regex` are configured in the same `header` configuration. | string | optional |
| `suffix` | Specifies a suffix value for the header key set in the `name` field. If the request header value ends with the `suffix` value, Consul applies the permission. Do not specify `suffix` if `present`, `exact`, `prefix`, or `regex` are configured in the same `header` configuration. | string | optional |
| `regex` | Specifies a regular expression pattern as the value for the header key set in the `name` field. If the request header value matches the regex, Consul applies the permission. Do not specify `regex` if `present`, `exact`, `prefix`, or `suffix` are configured in the same `header` configuration. The regex syntax is proxy-specific. If using Envoy, refer to the [re2 documentation](https://github.com/google/re2/wiki/Syntax) for details. | string | optional |
| `present` | Enables a match if the header configured in the `name` field appears in the request. Consul matches on any value as long as the header key appears in the request. Do not specify `present` if `exact`, `prefix`, `suffix`, `contains`, or `regex` are configured in the same `header` configuration. | boolean | optional |
| `Exact` | Specifies a value for the header key set in the `Name` field. If the request header value matches the `exact` value, Consul applies the permission. Do not specify `exact` if `present`, `prefix`, `suffix`, `contains`, or `regex` are configured in the same `header` configuration. | string | optional |
| `prefix` | Specifies a prefix value for the header key set in the `name` field. If the request header value starts with the `prefix` value, Consul applies the permission. Do not specify `prefix` if `present`, `exact`, `suffix`, `contains`, or `regex` are configured in the same `header` configuration. | string | optional |
| `suffix` | Specifies a suffix value for the header key set in the `name` field. If the request header value ends with the `suffix` value, Consul applies the permission. Do not specify `suffix` if `present`, `exact`, `prefix`, `contains`, or `regex` are configured in the same `header` configuration. | string | optional |
| `contains` | Specifies a contains value for the header key set in the `name` field. If the request header value includes the `contains` value, Consul applies the permission. Do not specify `contains` if `present`, `exact`, `prefix`, `suffix`, or `regex` are configured in the same `header` configuration. | string | optional |
| `regex` | Specifies a regular expression pattern as the value for the header key set in the `name` field. If the request header value matches the regex, Consul applies the permission. Do not specify `regex` if `present`, `exact`, `prefix`, `suffix`, or `contains` are configured in the same `header` configuration. The regex syntax is proxy-specific. If using Envoy, refer to the [re2 documentation](https://github.com/google/re2/wiki/Syntax) for details. | string | optional |
| `ignoreCase` | Ignores the case of the provided header value when matching with exact, prefix, suffix, or contains. Default is `false`. | boolean | optional |
| `invert` | Inverts the matching logic configured in the `header`. Default is `false`. | boolean | optional |
### `spec.sources[].type`

View File

@ -38,6 +38,10 @@ L4 intentions mediate the ability to establish new connections. Modifying an int
L7 intentions mediate the ability to issue new requests. When an intention is modified, requests received after the modification use the latest intention rules to enforce access. Changing a connection from `allow` to `deny` does not sever the connection, but doing so blocks new requests from being processed.
When using L7 intentions, we recommend that you review and update the [Mesh request normalization configuration](/consul/docs/connect/security#request-normalization-and-configured) to avoid unintended match rule circumvention. More details are available in the [Mesh configuration entry reference](/consul/docs/connect/config-entries/mesh#request-normalization).
When you use L7 intentions with header matching and it is possible for a header to contain multiple values, we recommend using `contains` or `regex` instead of `exact`, `prefix`, or `suffix`. For more information, refer to the [service intentions configuration entry reference](/consul/docs/connect/config-entries/service-intentions#spec-sources-permissions-http-header).
### Caching
The intentions for services registered with a Consul agent are cached locally on the agent. Supported proxies also cache intention data in their own configurations so that they can authorize inbound connections or requests without relying on the Consul agent. All actions in the data path of connections take place within the proxy.

View File

@ -26,6 +26,32 @@ of Consul.
## Checklist
### Request Normalization Configured for L7 Intentions
~> **Compatibility warning**: This feature is available as of Consul CE 1.20.1 and Consul Enterprise 1.20.1, 1.19.2, 1.18.3, and 1.15.15. We recommend upgrading to the latest version of Consul to take advantage of the latest features and improvements.
Atypical traffic patterns may interfere with the enforcement of L7 intentions. For
example, if a service makes request to a non-normalized URI path and Consul is not
configured to force path normalization, it becomes possible to circumvent path match rules. While a
default deny policy can limit the impact of this issue, we still recommend
that you review your current request normalization configuration. Normalization is critical to avoid unwanted
traffic, especially when using unrecommended security options such as a default allow intentions policy.
Consul adopts a default normalization mode that adheres to [RFC 3986](
https://tools.ietf.org/html/rfc3986#section-6), but additional options to enable stricter
normalization are available in the cluster-wide [Mesh configuration entry](
/consul/docs/connect/config-entries/mesh). We recommend reviewing these options and
enabling the strictest set that does not interfere with application traffic.
We also recommend that you review L7 intention header match rules for potential
issues with multiple header values. Refer to the [service intentions
configuration entry reference](/consul/docs/connect/config-entries/service-intentions#spec-sources-permissions-http-header)
for more information.
You do not need to enable request normalization if you are not using L7 intentions.
However, normalization may also benefit the use of other service mesh features that
rely on L7 attribute matching, such as [service routers](/consul/docs/connect/manage-traffic#routing).
### ACLs Enabled with Default Deny
Consul must be configured to use ACLs with a default deny policy. This forces
@ -43,6 +69,10 @@ this. **If ACLs are not enabled**, deny intentions will still be enforced, but a
may edit intentions. This renders the security of the created intentions
effectively useless.
The advantage of a default deny policy in combination with specific "allow" rules
is that at worst, a failure of intentions due to misconfiguration will result in
_denied_ traffic, rather than unwanted _allowed_ traffic.
### TCP and UDP Encryption Enabled
TCP and UDP encryption must be enabled to prevent plaintext communication

View File

@ -26,6 +26,10 @@ environment, but the general mechanisms for a secure Consul deployment revolve a
[authentication methods](/consul/docs/security/acl/auth-methods) can be used to enable trusted external parties to authorize
ACL token creation.
- **Intentions** - If in use, configure service intentions to use a default-deny policy. If L7 intentions are
in use, enable [Mesh request normalization](/consul/docs/connect/config-entries/mesh#request-normalization)
and review your [header match rules](/consul/docs/connect/config-entries/service-intentions#spec-sources-permissions-http-header) to prevent malformed requests from bypassing intentions.
- **Namespaces** <EnterpriseAlert inline /> - Read and write operations can be scoped to a logical namespace to restrict
access to Consul components within a multi-tenant environment.
@ -184,6 +188,13 @@ environment and adapt these configurations accordingly.
- **🏷 Namespace** <EnterpriseAlert inline /> - a named, logical scoping of Consul Enterprise resources, typically to
enable multi-tenant environments. Consul OSS clusters always operate within the "default" namespace.
- **Intentions** - Service intentions control traffic communication between services at the network layer (L4) and
application layer (L7). If in use, we strongly recommend configuring intentions to use a default-deny policy.
When L7 intentions are in use, review your configuration for [Mesh request normalization](/consul/docs/connect/config-entries/mesh#request-normalization)
and use the strictest set of options suitable to your environment. At minimum, we
recommend keeping path normalization enabled, because this default setting prevents requests that do not conform to [RFC 3986](
https://tools.ietf.org/html/rfc3986#section-6) from bypassing path match rules.
- **Gossip Encryption** - A shared, base64-encoded 32-byte symmetric key is required to [encrypt Serf gossip
communication](/consul/tutorials/security/gossip-encryption-secure?utm_source=consul.io&utm_medium=docs) within a cluster using
AES GCM. The key size determines which AES encryption types to use; 16, 24, or 32 bytes to select AES-128, AES-192,
@ -258,6 +269,10 @@ environment and adapt these configurations accordingly.
}
```
- **Customize Mesh HTTP Request Normalization** - If L7 intentions are in use, we recommend configuring request normalization to
avoid match rule circumvention. Other normalization options, such as dropping or rejecting headers with underscores,
may also be appropriate depending on your requirements. Review the options in the [Mesh configuration entry](/consul/docs/connect/config-entries/mesh#request-normalization) to determine the appropriate settings for your use case.
- **Customize Default Limits** - Consul has a number of builtin features with default connection limits that should be
tuned to fit your environment.

View File

@ -156,6 +156,10 @@ In Consul v1.15 and higher:
</CodeBlockConfig>
### Mesh traffic request path normalization enabled by default
As of Consul v1.15.15, inbound traffic to mesh proxies will have Envoy request [path normalization](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto#envoy-v3-api-field-extensions-filters-network-http-connection-manager-v3-httpconnectionmanager-normalize-path) applied by default. This should not interfere with the majority of service traffic, but can be disabled if needed by setting `http.incoming.request_normalization.insecure_disable_path_normalization` to `true` in the [global `mesh` configuration entry](/consul/docs/connect/config-entries/mesh#request-normalization). This setting is generally safe to change if not using L7 intentions with path matching.
## Consul 1.14.x
### Service Mesh Compatibility