consul/website/content/docs/connect/manage-traffic/fault-injection.mdx

---
layout: docs
page_title: Fault injection
description: Learn how to use fault injection in a Consul service mesh to inject artificial latency, response codes, or bandwidth limits for application testing.
---

# Fault Injection

This topic describes the process to inject different types of faults into the services in Consul's service mesh to help you test your network's resilience.

<EnterpriseAlert> This feature is available in Consul Enterprise. </EnterpriseAlert>

## Introduction

Consul allows you to configure fault injection filters to alter the responses from an upstream service. By injecting HTTP and gRPC statuses, bandwidth limits, or delays, users can test the resilience of their system to different unexpected issues.

Consul applies fault injection filters to an upstream service as part of its service defaults. For example, to simulate that an upstream returned a 500 HTTP status code 50 percent of the time, add an `Abort` configuration to the Envoy extensions block in the service default configuration for the upstream service.

The fault injection filters may be used individually, or in a combination of the three types by adding multiple blocks to the `Arguments.Config` section of the configuration.

## Requirements

Consul Enterprise v1.18.0 or later

## Inject delays

Specify the fault injection behavior in the [service defaults configuration entry](/consul/docs/connect/config-entries/service-defaults).

<Tabs>
  <Tab heading="VMs" group="vms">

To inject a delay fault when Consul runs on virtual machines, configure the following parameters:

1. `Arguments.Config.Delay.Duration`: The delay in milliseconds to use when injecting a delay type fault.
1. `Arguments.Config.Delay.Percentage`: The percentage of responses to inject the delay behavior into.

The following example configures the default behavior for a service named `billing`. This configuration injects a 100 ms delay into 50 percent of responses from the billing service.

```hcl
Kind = "service-defaults"
Name = "billing"
EnvoyExtensions = [
{
  Name = "builtin/fault-injection"
  Arguments = {
    Config = {
      Delay = {
        Duration = 100
        Percentage = 50
      }
    }
  }
}
]
```

  </Tab>
  <Tab heading="Kubernetes" group="k8s">

To inject a delay fault when Consul runs on Kubernetes, configure the following parameters:

1. `spec.arguments.config.delay.duration`: The delay in milliseconds to use when injecting a delay type fault.
1. `spec.arguments.config.delay.percentage`: The percentage of responses to inject the delay behavior into.

The following example configures the default behavior for a service named `billing`. This configuration injects a 100 ms delay into 50 percent of responses from the billing service.

```yaml
kind: ServiceDefaults
metadata:
  name: billing
spec:
  envoyExtensions:
  - name: "builtin/fault-injection"
  arguments:
    config:
      delay:
        duration: 100
        percentage: 50
```

  </Tab>
</Tabs>


Refer to the [service defaults configuration entry reference](/consul/docs/connect/config-entries/service-defaults) for additional specifications and example configurations.

## Inject statuses

<Tabs>
  <Tab heading="VMs" group="vms">
To inject a status code when aborting a response, configure the following parameters: 

1. `Arguments.Config.Abort.HttpStatus`: The HTTP status code to inject into responses when injecting an abort type fault. You may specify either a HTTP status or a gRPC status but not both.
1. `Arguments.Config.Abort.GrpcStatus`: The gRPC status to inject into responses when injecting an abort type fault. You may specify either a HTTP status or a gRPC status but not both.
1. `Arguments.Config.Abort.Percentage`: The percentage of responses to inject the abort behavior into.

The following example configures the default behavior for a service named `billing`. This configuration injects an HTTP status code of 503 into 5 percent of responses from the billing service.

```hcl
Kind = "service-defaults"
Name = "billing"
EnvoyExtensions = [
{
  Name = "builtin/fault-injection"
  Arguments = {
    Config = {
      Abort = {
        HttpStatus = 503
        Percentage = 5
      }
    }
  }
}
]
```

  </Tab>
  <Tab heading="Kubernetes" group="k8s">

To inject a status code when aborting a response, configure the following parameters: 

1. `spec.arguments.config.abort.httpStatus`: The HTTP status code to inject into responses when injecting an abort type fault. You may specify either a HTTP status or a gRPC status but not both.
1. `spec.arguments.config.abort.grpcStatus`: The gRPC status to inject into responses when injecting an abort type fault. You may specify either a HTTP status or a gRPC status but not both.
1. `spec.arguments.config.abort.percentage`: The percentage of responses to inject the abort behavior into.

The following example configures the default behavior for a service named `billing`. This configuration injects an HTTP status code of 503 into 5 percent of responses from the billing service.

```yaml
kind: ServiceDefaults
metadata:
  name: billing
spec:
  envoyExtensions:
  - name: "builtin/fault-injection"
  arguments:
    config:
      abort:
        httpStatus: 503
        percentage: 5
```

  </Tab>
</Tabs>

Refer to the [service defaults configuration entry reference](/consul/docs/connect/config-entries/service-defaults) for additional specifications and example configurations.

## Inject bandwidth limit

<Tabs>
  <Tab heading="VMs" group="vms">

To inject a rate limit into a response, configure the following parameters: 

1. `Arguments.Config.Bandwidth.Limit`: The amount of data allowed through the filter when applied. This value is specified in KiB/s. When the limit is exceeded, requests, operations, or connections will be subjected to rate limiting.
1. `Arguments.Config.Bandwidth.Percentage`: The percent of responses to inject the bandwidth limit behavior into.

The following example configures the default behavior for a service named `billing`. This configuration limits the bandwidth of outgoing requests to 100 KiB/s for 50 percent of responses from the billing service.

```hcl
Kind = "service-defaults"
Name = "billing"
EnvoyExtensions = [
{
  Name = "builtin/fault-injection"
  Arguments = {
    Config = {
      Bandwidth = {
        Limit = 100
        Percentage = 50
      }
    }
  }
}
]
```

  </Tab>
  <Tab heading="Kubernetes" group="k8s">

To inject a rate limit into a response, configure the following parameters: 

1. `spec.arguments.config.bandwidth.limit`: The amount of data allowed through the filter when applied. This value is specified in KiB/s. When the limit is exceeded, requests, operations, or connections will be subjected to rate limiting.
1. `spec.arguments.config.bandwidth.percentage`: The percent of responses to inject the bandwidth limit behavior into.

The following example configures the default behavior for a service named `billing`. This configuration limits the bandwidth of outgoing requests to 100 KiB/s for 50 percent of responses from the billing service.

```yaml
kind: ServiceDefaults
metadata:
  name: billing
spec:
  envoyExtensions:
  - name: "builtin/fault-injection"
  arguments:
  config:
    bandwidth:
      limit: 100
      percentage: 50
```

  </Tab>
</Tabs>

Refer to the [service defaults configuration entry reference](/consul/docs/connect/config-entries/service-defaults) for additional specifications and example configurations.