AWS ECS(EC2 Container service)

I recently tried out AWS ECS(EC2 Container service). In this blog, I will provide overview of EC2 container service, some hands-on stuff I tried and my experiences with ECS. AWS ECS is available as a preview currently with only CLI interface. I assume general availability with web interface will be available soon. Amazon does not charge separately for the ECS service, instances used for containers will be charged appropriately. I have a AWS free-tier account and I was able to try the ECS with the free-tier account.

Pre-requisites:

  • Familiarity with Containers and Docker. You can refer to my blog on Docker series.
  • Familiarity with AWS services. You can refer to my previous blogs on AWS overview and how to access AWS services.
  • For the hands-on stuff, you need to have atleast a free-tier account with AWS. AWS cli needs to be installed since ECS is available only with a CLI interface currently.

Overview of ECS:

I had written a blog earlier on Docker Orchestration. ECS is a Docker Container Orchestration service that provides the following functionalities.

  • Allows to deploy Containers across a cluster of hosts using either AWS scheduler or integrate with third party schedulers like Mesos.
  • Allows management of an application that spans multiple containers rather than just managing containers.
  • Provides high availability for Containers by monitoring Container health.

Following are components of ECS:

  • Cluster – Logical entity that contains multiple EC2 instances. Instances within the container can be of different types.
  • Container instances – Each instance is an EC2 instance with Docker and Container agent pre-installed and is available as AWS AMI image. Container agent registers itself to the cluster and it monitors health of the Containers in the host. AWS is open-sourcing the Container agent project and its available in github.
  • Tasks – Tasks contains the application definition, resources needed and it can span multiple containers. Task definitions are specified in json format.
  • ECS Master – This manages everything. ECS master is responsible for scheduling the containers and also for talking to the container agent to get the health of container instances. When ECS master schedules the containers, it takes into account resources needed for the container and the resources available in the container instance.

ECS hands-on:

Most of the steps below are as mentioned in AWS ECS developer guide.

Initial steps:

  • Create IAM user and login as IAM user. IAM user allows to give more granular control of AWS resources based on user login.
  • Create IAM policy that allows container instances to be part of the cluster. Create a role “AmazonECSContainerInstanceRole” and associate the IAM policy. When creating the Container instances, we will use the role created in this step.
  • Create keypair and security group. Keypair will be used to ssh to the container instance. Security group controls the networking access to the Container. As a minimum, ssh access is needed to the Container instance.
  • Create 2 EC2 instances using custom AMI image “amzn-ami-2014.09.1-amazon-ecs-optimized-preview3 (ami-4d3c1a7d)”, IAM policy “AmazonECSContainerInstanceRole” and keypair, security group created in previous steps. Instance size could be anything, I chose t2.micro. i created the instances in us-west-2(Oregon). Only 2 regions support ECS currently.
  • Default cluster and VPC group was already created, there is no need to change this unless we need to create custom clusters or VPC groups.

Lets list the clusters. We can see 1 default cluster.

$ aws ecs list-clusters
{
    "clusterArns": [
        "arn:aws:ecs:us-west-2:173760706945:cluster/default"
    ]
}

List the container instances that are part of the default cluster. Here, we see the 2 EC2 instances that are part of the default cluster.

$ aws ecs list-container-instances --cluster default
{
    "containerInstanceArns": [
        "arn:aws:ecs:us-west-2:173760706945:container-instance/2431ade6-f390-4297-a941-a013906a14b2", 
        "arn:aws:ecs:us-west-2:173760706945:container-instance/daf0011b-5c88-4756-8604-2fc70c238f8b"
    ]
}

To get more details on a single container instance, we can do the following command. I have pasted part of the output which shows the cpu and memory available for that container instance. The numbers are part of t2.micro EC2 instance.

$ aws ecs describe-container-instances --cluster default --container-instances 2431ade6-f390-4297-a941-a013906a14b2
{
    "failures": [], 
    "containerInstances": [
        {
            "status": "ACTIVE", 
            "registeredResources": [
                {
                    "integerValue": 1024, 
                    "longValue": 0, 
                    "type": "INTEGER", 
                    "name": "CPU", 
                    "doubleValue": 0.0
                }, 
                {
                    "integerValue": 996, 
                    "longValue": 0, 
                    "type": "INTEGER", 
                    "name": "MEMORY", 
                    "doubleValue": 0.0
                }, 

Lets ssh to 1 of the container instances and see what containers are present. ECS agent runs by default as a container.

]$ sudo docker ps
CONTAINER ID        IMAGE                            COMMAND             CREATED             STATUS              PORTS                        NAMES
4f7bac4f5e85        amazon/amazon-ecs-agent:latest   "/agent"            3 hours ago         Up 3 hours          127.0.0.1:51678->51678/tcp   ecs-agent         

Lets look at the current tasks registered and then register a new task. Below, we see multiple tasks already registered. I had these tasks earlier. As of now, I dont see a way to deregister a task, so I have left it like this.

smakam14@sreeubuntu14-VirtualBox:~$ aws ecs list-task-definitions
{
    "taskDefinitionArns": [
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:1", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:2", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:3", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:4", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:5", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:6", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:7", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/hello_world:1", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/hello_world:2", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/sleep360:1", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/sleep360:2"
    ]
}

Lets create a new task like below with busybox container that does echo of “hello world” indefinitely.

{
  "containerDefinitions": [
    {
      "name": "sleep",
      "image": "busybox",
      "cpu": 10,
      "entryPoint": [
        "sh",
        "-c"
      ],
      "command": ["while true; do echo Hello World; sleep 1; done"],
      "memory": 10,
      "essential": true
    }
  ],
  "family": "busy1"
}

Lets register this task:

$ aws ecs register-task-definition --cli-input-json file://busy1.json
{
    "taskDefinition": {
        "volumes": [], 
        "taskDefinitionArn": "arn:aws:ecs:us-west-2:173760706945:task-definition/busyecho:1", 
        "containerDefinitions": [
            {
                "environment": [], 
                "name": "busyecho", 
                "mountPoints": [], 
                "image": "busybox", 
                "cpu": 10, 
                "portMappings": [], 
                "entryPoint": [
                    "sh", 
                    "-c"
                ], 
                "memory": 10, 
                "command": [
                    "while true; do echo Hello World; sleep 1; done"
                ], 
                "essential": true, 
                "volumesFrom": []
            }
        ], 
        "family": "busyecho", 
        "revision": 1
    }
}

Now, when we list the task definitions, we should see our new task “busyecho:1”

$ aws ecs list-task-definitions
{
    "taskDefinitionArns": [
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:1", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:2", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:3", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:4", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:5", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:6", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busy1:7", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/busyecho:1", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/hello_world:1", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/hello_world:2", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/sleep360:1", 
        "arn:aws:ecs:us-west-2:173760706945:task-definition/sleep360:2"
    ]
}

Lets spawn 4 instances of the task created above.

$ aws ecs run-task --cluster default --task-definition busyecho:1 --count 4

Lets list the tasks running. We can see 4 tasks.

$ aws ecs list-tasks --cluster default
{
    "taskArns": [
        "arn:aws:ecs:us-west-2:173760706945:task/4612d2e7-365c-4efb-92f0-3c2ff24015e9", 
        "arn:aws:ecs:us-west-2:173760706945:task/48653f79-307c-4a1f-b35b-1268ef24f212", 
        "arn:aws:ecs:us-west-2:173760706945:task/4e6574d6-ea12-423e-9e28-d946288e345a", 
        "arn:aws:ecs:us-west-2:173760706945:task/53309ae4-4192-4476-aaa2-5754c6cfedf1"
    ]
}

ECS will automatically load balance these tasks into the different container instances in the cluster. In this case, I saw 2 tasks running in each EC2 container instance.

$ sudo docker ps
CONTAINER ID        IMAGE                            COMMAND                CREATED             STATUS              PORTS                        NAMES
4876af5568c1        busybox:buildroot-2014.02        "sh -c 'while true;    2 minutes ago       Up 2 minutes                                     ecs-busyecho-1-busyecho-cc81f3a9e0dcfb937000   
1858633b909f        busybox:buildroot-2014.02        "sh -c 'while true;    2 minutes ago       Up 2 minutes                                     ecs-busyecho-1-busyecho-88e6b9b893a8f8eceb01   
4f7bac4f5e85        amazon/amazon-ecs-agent:latest   "/agent"               3 hours ago         Up 3 hours          127.0.0.1:51678->51678/tcp   ecs-agent                            
$ sudo docker ps 
CONTAINER ID        IMAGE                            COMMAND                CREATED             STATUS              PORTS                        NAMES
4bb86ec0a261        busybox:buildroot-2014.02        "sh -c 'while true;    2 minutes ago       Up 2 minutes                                     ecs-busyecho-1-busyecho-e0cfc18adda7e6eff401   
b7b75c72d3b3        busybox:buildroot-2014.02        "sh -c 'while true;    2 minutes ago       Up 2 minutes                                     ecs-busyecho-1-busyecho-f484bfdb90fd97e26100   
ebc89fa8d0f9        amazon/amazon-ecs-agent:latest   "/agent"               3 hours ago         Up 3 hours          127.0.0.1:51678->51678/tcp   ecs-agent              

By checking the container logs, we can confirm that the container is running and giving expected output.

$ docker logs 4bb86ec0a261
Hello World
Hello World
.

I also tried a task with multiple containers like a wordpress application. Make sure that the total memory does not exceed memory available in a single container instance. Below, we can see that the wordpress application contains 2 containers, web container and db container. web container links to db container and exposes port 80. The “essential” flag set to true means that the task can survive only if both containers are active.

$ cat wordpress.json 
{
  "containerDefinitions": [
    {
      "name": "wordpress",
      "links": [
        "mysql"
      ],
      "image": "wordpress",
      "essential": true,
      "portMappings": [
        {
          "containerPort": 80,
          "hostPort": 80
        }
      ],
      "memory": 250,
      "cpu": 10
    },
    {
      "environment": [
        {
          "name": "MYSQL_ROOT_PASSWORD",
          "value": "password"
        }
      ],
      "name": "mysql",
      "image": "mysql",
      "cpu": 10,
      "memory": 250,
      "essential": true
    }
  ],
  "family": "hello_world"
}

I have already registered this task. To run the task:

aws ecs run-task --cluster default --task-definition hello_world:2 --count 1

Lets list the containers in the Container instance. We can see the wordpress http container and mysql db container.

$ sudo docker ps
CONTAINER ID        IMAGE                            COMMAND                CREATED              STATUS                  PORTS                        NAMES
013f3e56207f        wordpress:4                      "/entrypoint.sh apac   1 seconds ago        Up Less than a second   0.0.0.0:80->80/tcp           ecs-hello_world-2-wordpress-f8ceb4ac99b19dd06b00   
3c4b9181b9da        mysql:5                          "/entrypoint.sh mysq   About a minute ago   Up About a minute       3306/tcp                     ecs-hello_world-2-mysql-84b2f0b1e5a4ac94ee01       
4bb86ec0a261        busybox:buildroot-2014.02        "sh -c 'while true;    10 minutes ago       Up 10 minutes                                        ecs-busyecho-1-busyecho-e0cfc18adda7e6eff401       
b7b75c72d3b3        busybox:buildroot-2014.02        "sh -c 'while true;    10 minutes ago       Up 10 minutes                                        ecs-busyecho-1-busyecho-f484bfdb90fd97e26100       
ebc89fa8d0f9        amazon/amazon-ecs-agent:latest   "/agent"               3 hours ago          Up 3 hours              127.0.0.1:51678->51678/tcp   ecs-agent                              

At this point, i was able to access the WordPress application after I enabled the port 80 access from the outside world in the security group.

To stop the wordpress application, do this:

$ aws ecs stop-task --cluster default --task 0b3ff5fc-d1fc-4e80-ace0-5fe67077c706

Lets list the running tasks now and we dont see the wordpress task:

$ aws ecs list-tasks --cluster default
{
    "taskArns": [
        "arn:aws:ecs:us-west-2:173760706945:task/4612d2e7-365c-4efb-92f0-3c2ff24015e9", 
        "arn:aws:ecs:us-west-2:173760706945:task/48653f79-307c-4a1f-b35b-1268ef24f212", 
        "arn:aws:ecs:us-west-2:173760706945:task/4e6574d6-ea12-423e-9e28-d946288e345a", 
        "arn:aws:ecs:us-west-2:173760706945:task/53309ae4-4192-4476-aaa2-5754c6cfedf1"
    ]
}

If we look at the container instances, we would see that both the containers(wordpress web and db) would be removed.

Issues faced:

  1. After I registered a task using “aws ecs register-task-definition”, I did not see an option to deregister the tasks.
  2. When I create the instance, I see that the container instance is part of default cluster. I am able to register tasks and run it and it works fine. When I stop and start the instance again, I see that the instance does not get registered to the default cluster. I tried different things like stopping and starting the container agent again(in the instance) and it does not seem to work. The only way I can get going is to launch a new instance…
  3. I had 2 container instances as part of default cluster. When I stopped 1 of the container instances, I was expecting that using the container agent, ecs cluster would detect it and move the task to other container instance. That did not work. Infact, I could not see what happened to my task.

I have posted these issues to AWS forum(https://forums.aws.amazon.com/thread.jspa?threadID=172676) and waiting for the response. I am assuming that when ECS is available after preview, these issues would have been addressed. (Update: Based on the response received from AWS forum, issue 2 is a bug and has been addressed now)

Overall, my experience with ECS service has been very good. Its pretty intuitive and it will be very powerful when integrated with other AWS services.

References:

Advertisements

5 thoughts on “AWS ECS(EC2 Container service)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s