mirror of https://github.com/hashicorp/consul
277 lines
13 KiB
Markdown
277 lines
13 KiB
Markdown
---
|
|
layout: docs
|
|
page_title: Installing Consul on AWS ECS using Terraform
|
|
description: >-
|
|
Install Consul Service Mesh on AWS ECS with Terraform (Elastic Container Service).
|
|
---
|
|
|
|
# Installation with Terraform
|
|
|
|
This topic describes how to use the [`mesh-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) Terraform module to launch your application in AWS ECS as part of Consul service mesh. If you do not use Terraform, see the [Manual Installation](/docs/ecs/manual-installation) page to install Consul on ECS without Terraform.
|
|
|
|
This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples.
|
|
|
|
## Overview
|
|
|
|
This topic describes the following procedure:
|
|
|
|
1. Create Terraform configuration files for the necessary components:
|
|
|
|
* [ECS task definition](#using-the-mesh-task-module): Use the `mesh-task` module to create an ECS task definition for Consul on ECS
|
|
* [ECS service](#ecs-service): Use the `aws_ecs_service` resource to create an ECS service that schedules service mesh tasks to run on ECS
|
|
|
|
2. [Run Terraform](#running-terraform) to deploy the resources in AWS
|
|
|
|
## Prerequisites
|
|
|
|
* You should have some familiarity with using Terraform. Refer to the [Terraform documentation](https://www.terraform.io/docs) to learn about infrastructure as code and how to get started with Terraform.
|
|
* You should also be familiar with AWS ECS before following these instructions. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for details.
|
|
|
|
## Using the Mesh Task Module
|
|
|
|
To run an application in ECS with Consul service mesh, you must create an ECS task definition, which includes your application container(s)
|
|
and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container.
|
|
|
|
The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) will automatically include the necessary sidecar containers.
|
|
|
|
The following example shows a Terraform configuration file that creates a task definition with an application container called `example-client-app` in a file called `mesh-task.tf`:
|
|
|
|
<CodeBlockConfig filename="mesh-task.tf">
|
|
|
|
```hcl
|
|
module "my_task" {
|
|
source = "hashicorp/consul-ecs/aws//modules/mesh-task"
|
|
version = "<latest version>"
|
|
|
|
family = "my_task"
|
|
container_definitions = [
|
|
{
|
|
name = "example-client-app"
|
|
image = "docker.io/org/my_task:v0.0.1"
|
|
essential = true
|
|
portMappings = [
|
|
{
|
|
containerPort = 9090
|
|
hostPort = 9090
|
|
protocol = "tcp"
|
|
}
|
|
]
|
|
cpu = 0
|
|
mountPoints = []
|
|
volumesFrom = []
|
|
}
|
|
]
|
|
|
|
port = 9090
|
|
retry_join = ["<address of the Consul server>"]
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
The following fields are required. Refer to the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs) for a complete reference.
|
|
|
|
| Input Variable | Type | Description |
|
|
| ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `source` | string | Must be set to the source location of the `mesh-task` module, `hashicorp/consul-ecs/aws//modules/mesh-task`. |
|
|
| `version` | string | Must be set to the version of the `mesh-task` module. |
|
|
| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. |
|
|
| `container_definitions` | list | This is the list of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions) for the task definition. This is where you include your application containers. |
|
|
| `essential` | boolean | Must be `true` to ensure the health of your application container affects the health status of the task. |
|
|
| `port` | integer | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. |
|
|
| `retry_join` | list | The is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. |
|
|
|
|
### Running Terraform
|
|
|
|
You will need to run Terraform to create the task definition.
|
|
|
|
Save the Terraform configuration for the task definition to a file, such as `mesh-task.tf`.
|
|
You should place this file in a directory alongside other Terraform configuration files for your project.
|
|
|
|
The `mesh-task` module requires the AWS Terraform provider. The following example shows how to include
|
|
and configure the AWS provider in a file called `provider.tf`. Refer to the [AWS Terraform provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)
|
|
documentation for complete configuration details.
|
|
|
|
<CodeBlockConfig filename="provider.tf">
|
|
|
|
```hcl
|
|
terraform {
|
|
required_providers {
|
|
aws = {
|
|
source = "hashicorp/aws"
|
|
version = "<latest version>"
|
|
}
|
|
}
|
|
}
|
|
|
|
provider "aws" {
|
|
region = "<AWS region>"
|
|
...
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
Additional AWS resources for your project can be included in additional Terraform configuration files
|
|
in the same directory. The following example shows a basic project directory:
|
|
|
|
```shell-session
|
|
$ ls
|
|
mesh-task.tf
|
|
provider.tf
|
|
...
|
|
```
|
|
|
|
Terraform should be run in your project directory as follows.
|
|
|
|
* Run `terraform init` first to download dependencies, such as Terraform providers
|
|
* Run `terraform apply` to have Terraform create AWS resources, such as the task definition from the `mesh-task` module.
|
|
|
|
Terraform automatically reads all files in the current directory that have a `.tf` file extension.
|
|
Refer to the [Terraform documentation](https://www.terraform.io/docs) for more information and Terraform best practices.
|
|
|
|
## ECS Service
|
|
|
|
[ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) are one of the most common
|
|
ways to start tasks using a task definition.
|
|
|
|
To define an ECS service, reference the `mesh-task` module's `task_definition_arn` output value
|
|
in your `aws_ecs_service` resource. The following example shows how to include the service in the `mesh-task.tf` file.
|
|
|
|
<CodeBlockConfig filename="mesh-task.tf" highlight="6-12">
|
|
|
|
```hcl
|
|
module "my_task" {
|
|
source = "hashicorp/consul-ecs/aws//modules/mesh-task"
|
|
...
|
|
}
|
|
|
|
resource "aws_ecs_service" "my_task" {
|
|
name = "my_task_service"
|
|
task_definition = module.my_task.task_definition_arn
|
|
launch_type = "FARGATE"
|
|
propagate_tags = "TASK_DEFINITION"
|
|
...
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
This is a partial configuration to highlight some important fields.
|
|
See the [`aws_ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) documentation for a complete reference.
|
|
|
|
| Input Variable | Type | Description |
|
|
| ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------- |
|
|
| `name` | string | The name of the ECS service. This is required by AWS but is not used by Consul service mesh. |
|
|
| `task_definition` | string | The task definition used to start tasks. Set this to the task definition ARN returned by the `mesh-task` module. |
|
|
| `launch_type` | string | The launch type. Consul on ECS supports the `FARGATE` and `EC2` launch types. |
|
|
| `propagate_tags` | string | This must be set to `TASK_DEFINITION` so that tags added by `mesh-task` to the task definition are copied to tasks. |
|
|
|
|
After including the ECS service in your Terraform configuration, run `terraform apply`
|
|
from your project directory to create the ECS service resource. The ECS service will
|
|
soon start your application in a task. The task will automatically register itself
|
|
into the Consul service catalog during startup.
|
|
|
|
-> **NOTE:** If your tasks run in a public subnet, they must have `assign_public_ip = true`
|
|
in their [`network_configuration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#network_configuration) block so that ECS can pull the Docker images.
|
|
|
|
## Routing
|
|
|
|
Now that your tasks are registered in the mesh, you're able to use the service
|
|
mesh to route between them.
|
|
|
|
In order to make calls through the service mesh, you must configure the sidecar
|
|
proxy to listen on a different port for each upstream service your application
|
|
needs to call. You then must modify your application to make requests to the sidecar
|
|
proxy on that port.
|
|
|
|
For example, if your application `web` makes calls to another application called `backend`, then you would first configure the `mesh-task` module's upstream(s):
|
|
|
|
```hcl
|
|
module "web" {
|
|
family = "web"
|
|
upstreams = [
|
|
{
|
|
destinationName = "backend"
|
|
localBindPort = 8080
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
| Input Variable | Type | Description |
|
|
| ----------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `destinationName` | string | The name of the upstream service, as it is registered in the Consul service catalog. |
|
|
| `localBindPort` | integer | Requests to this port will be forwarded by the proxy to the upstream service. This must be an unused port, but does not need to match the upstream service port. |
|
|
|
|
If you have multiple upstream services they each need to be listed here.
|
|
|
|
Next, configure your application to make requests to `localhost:8080` when
|
|
it wants to call the `backend` service.
|
|
|
|
For example, if your service allows configuring the URL for `backend` via the
|
|
`BACKEND_URL` environment variable, you would set:
|
|
|
|
```hcl
|
|
module "web" {
|
|
family = "web"
|
|
upstreams = [
|
|
{
|
|
destinationName = "backend"
|
|
localBindPort = 8080
|
|
}
|
|
]
|
|
container_definitions = [
|
|
{
|
|
name = "web"
|
|
environment = [
|
|
{
|
|
name = "BACKEND_URL"
|
|
value = "http://localhost:8080"
|
|
}
|
|
]
|
|
...
|
|
}
|
|
]
|
|
...
|
|
}
|
|
```
|
|
|
|
## Bind Address
|
|
|
|
To ensure that your application only receives traffic through the service mesh,
|
|
you must change the address that your application is listening on to only the loopback address
|
|
(also known as `localhost`, `lo`, and `127.0.0.1`)
|
|
so that only the sidecar proxy running in the same task can make requests to it.
|
|
|
|
If your application is listening on all interfaces, e.g. `0.0.0.0`, then other
|
|
applications can call it directly, bypassing its sidecar proxy.
|
|
|
|
Changing the listening address is specific to the language and framework you're
|
|
using in your application. Regardless of which language/framework you're using,
|
|
it's a good practice to make the address configurable via environment variable.
|
|
|
|
For example in Go, you would use:
|
|
|
|
```go
|
|
s := &http.Server{
|
|
Addr: "127.0.0.1:8080",
|
|
...
|
|
}
|
|
log.Fatal(s.ListenAndServe())
|
|
```
|
|
|
|
In Django you'd use:
|
|
|
|
```bash
|
|
python manage.py runserver "127.0.0.1:8080"
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
- Follow the [Secure Configuration](/docs/ecs/secure-configuration) to get production-ready.
|
|
- Now that your applications are running in the service mesh, read about
|
|
other [Service Mesh features](/docs/connect).
|
|
- View the [Architecture](/docs/ecs/architecture) documentation to understand
|
|
what's going on under the hood.
|