Microservices Infrastructure using Mantl

Mantl is an Open source project from Cisco and it provides an integrated solution to deploy distributed Microservices. Any company deploying Microservices has to integrate different components before the solution becomes production ready. Mantl makes it easier by integrating the different components and providing the glue software that integrates the components. In this blog, I will cover the following:

  • Distributed Microservice infrastructure components and the need for Mantl.
  • Mantl Architecture.
  • Mantl installation using Vagrant
  • Mantl installation using AWS public cloud

Microservices infrastructure

Following are typical components in Container based Microservices infrastructure:

  • Physical infrastructure – this can either be public or private cloud
  • OS – This can either be Linux flavors or Container optimized OS like CoreOS, Rancher, DCOS.
  • Container runtime – This can either be Docker, Rkt, LXC
  • Container orchestration – Kubernetes, Mesos, Swarm are possible choices
  • Service discovery – Consul, etcd, Zookeeper are possible choices
  • Logging and debugging – Logstash, LogEntries are possible choices
  • Networking – Weave, Calico can be used as networking plugin. Kubernetes and Mesos provide their own networking solution.
  • Storage – Local or distributed storage can be used. Since services can move across nodes in the cluster, the storage solution needs to cater to that need. Ceph and GlusterFS are choices for distributed storage.

Each customer deploying Microservices has to put these different components together and get them working. The goal of Mantl project is to provide an integrated Microservices infrastructure solution where they provide the glue software to stitch the different components together. Following are the current set of core components that Mantl integrates currently:

  • Container runtime – Docker
  • Orchestration – Mesos, Marathon (Kubernetes is planned soon)
  • Service discovery – Consul
  • Logging – ELK, Logstash
  • Networking – Calico
  • Storage – GlusterFS
  • Physical infrastructure – cloud provider or private cloud

Mantl Architecture

Following diagram from Mantl website describes the high level architecture of Mantl.

  • mantl1

Following are some notes on the Mantl architecture:

  • Mantl provides glue software to integrate the software components together. For example, mesos-consul and marathon-consul are glue software needed for Mesos and Marathon to register new services.
  • There are three types of nodes that Mantl provides. Control node runs the control software like Mesos leader, consul server, Docker , Marathon, Logstash and manages the resource nodes. Resource nodes run Mesos follower, consul agent, Docker, Logstash and runs the application Containers. Edge node is responsible for load balancing traffic to the services running in the Resource node.
  • Mantl uses Ansible for provisioning all the software components. Ansible playbooks and roles are used to simplify the provisioning.
  • VMs are provisioned using Hashicorp Terraform. VMs can be from any cloud provider or present in local data center.

Trying out Mantl

Mantl using Vagrant provider

I deployed Mantl using Packet’s baremetal cloud. I used their Type 0 server, this has only 4GB memory which can be used only for trial purposes. Following are the summary of steps:

  1. Create Type 0 baremetal cloud server in Packet with Ubuntu OS.
  2. Install general pre-requisite software that is needed for Mantl.
  3. Install Mantl specific pre-requisites.
  4. Setup security configurations.
  5. Provision environment using “vagrant up”.

General pre-requisites:
Following are the list of sofware that I needed before installing Mantl:

apt-get install -y git
apt-get install -y python-pip
wget https://releases.hashicorp.com/vagrant/1.8.1/vagrant_1.8.1_x86_64.deb
dpkg -i vagrant_1.8.1_x86_64.deb
wget http://download.virtualbox.org/virtualbox/5.0.16/virtualbox-5.0_5.0.16-105871~Ubuntu~trusty_amd64.deb
dpkg -i virtualbox-5.0_5.0.16-105871~Ubuntu~trusty_amd64.deb
# install dep
apt-get install -f
/etc/init.d/vboxdrv setup
apt-get install -y tightvncserver
apt-get install -y xfce4 xfce4-goodies
vncserver -geometry 2880x1620
apt-get install -y python-dev
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
dpkg -i google-chrome-stable_current_amd64.deb
# install dep
apt-get install -f

Following set of commands can be used to install Mantl, setup security configuration and provision the VMs

git clone https://github.com/CiscoCloud/mantl.git
cd mantl
pip install -r requirements.txt
vagrant up

The default Vagrantfile specifies creation of 1 control node and 1 resource node. I modified Vagrantfile to create 2 resource nodes before doing “Vagrant up”.
Following output shows the 3 nodes(1 control, 2 resource/worker) successfully running:

root@mantl:~/mantl# vagrant status
No vagrant-config.yml found, using defaults 
Current machine states:

worker-001                running (virtualbox)
worker-002                running (virtualbox)
control-01                running (virtualbox)

We can access the Controller using controller IP address which by default is
Following image shows two worker/slave nodes running inside Mesos:


Sample hello-world application

To check that the cluster is working, Mantl provides some sample applications and they are available in mantl/examples. I tried the hello-world example that deploys two instances of “hello-world” container. Following is the manifest file for this job:

  "container": {
    "type": "DOCKER",
    "docker": {
      "network": "BRIDGE",
      "image": "keithchambers/docker-hello-world",
      "portMappings": [
          "containerPort": 80,
          "protocol": "tcp"
  "id": "hello-world",
  "instances": 2,
  "cpus": 0.1,
  "mem": 128

To create this sample application, we can execute the following command from examples directory:

curl -k -X POST -H "Content-Type: application/json" -u "admin:mantl" -d@"hello-world/hello-world.json" ""

Following image shows the running applications inside Mesos. It shows that hello-world is running with 2 instances.


Following image shows the 2 running hello-world Container instances:


Following image shows the services that are automatically discovered by Consul and this includes hello-world.


Issues faced:

I had some issues when provisioning using Vagrant with respect to Consul service and I raised a case here. The workaround is mentioned in the case which is to redo the provisioning after reload using  “vagrant reload –provision”.

Mantl using AWS Cloud provider

Mantl can be installed in AWS, Google cloud, Digitalocean and also in Openstack. I tried installing Mantl in AWS cloud. AWS infrastructure steps are captured here. Following are the summary of the steps:

  • Create IAM user in AWS if not existing. It is preferable to not use root user.
  • Setup IAM user with administrator access by attaching the administrator policy. It is possible to have policy with lesser privileges, I had issues with just this “AmazonEC2FullAccess” policy.
  • Create ~/.aws/credentials file with access key and password for IAM user.
  • Install Ansible from here.
  • Install Terraform from here.
  • Copy “terraform/aws.sample.tf” to “mantl/aws.tf” and make necessary changes to “aws.tf”. In my case, I modified “aws.tf” to create 3 control nodes, 2 worker nodes, 2 edge nodes and selected instance type as “t2.medium”.
  • If DNS based access is needed, we need to create a Route53 record and update sub-domain and Zoneid in “aws.tf” file.
  • Provision the infrastructure using “terraform apply”. This will create all the nodes, apply necessary security policies etc. This step will not install Mantl related software.
  • Setup security configuration by running “security-setup” script.
  • Upgrade packages using “ansible-playbook playbooks/upgrade-packages.yml”
  • Copy “sample.yml” to “mantl.yml”. This file describes the services to be started on control, worker and edge nodes. I left this with default configuration.
  • Install Mantl using “ansible-playbook -e @security.yml mantl.yml”

Issues faced:
I got this error the first time I started Mantl installation using Ansible:

NOTIFIED: [handlers | wait for consul to listen] ****************************** 
failed: [mantl-control-01] => {"changed": true, "cmd": ["/usr/local/bin/consul-wait-for-leader.sh"], "delta": "0:00:35.035517", "end": "2016-03-25 07:41:51.606055", "rc": 1, "start": "2016-03-25 07:41:16.570538", "warnings": []}
stdout: No Consul leader elected in 30 seconds

I had to rerun Ansible script multiple times to avoid this issue.

Once Mantl installation is successful, we can access the control node either using the IP address or using DNS name if DNS is setup. Following is the front webpage showing the different services.mantl6

I started hello-world and Minecraft examples and following Marathon UI shows the different services started:


Overall, Mantl seems like a great effort to create an integrated solution that saves a lot of time for enterprises looking to deploy Microservices using Containers. Mantl uses Mesos as the Orchestration layer. Currently, Mantl is working with Kubernetes to help those customers who prefer to use Kubernetes instead of Mesos as the Orchestration layer. It will be interesting to see how flexible Mantl solution will allow swapping of components. For example, if we prefer a different service discovery, orchestration or logging solution, customers should be able to specify this in a configuration file and Mantl should add glue layer dynamically. This would be a perfect solution for customers who prefer components based on their environment, but would like to not add the glue by themselves.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s