Category Archives: orchestration

Comparing Docker compose versions

In this blog, I have captured some of my learnings on Docker compose files and how they differ between versions. Docker compose is a tool used for defining and running multi-container Docker applications. I have used the famous multi-container voting application to illustrate the differences with compose versions.

Following are some questions that I have to tried to answer in this blog:

  • What is the difference between Compose versions 1, 2 and 3?
  • What is the difference between compose, stack and dab formats?
  • What are different ways to run compose files with different compose versions?
  • How does “docker stack deploy” really work?

Compose versions:

Following table captures the main differences between Compose versions:

Continue reading Comparing Docker compose versions


Docker Compose and Interworking of Docker Machine, Swarm, Compose

This is a continuation of my previous 2 blogs on Docker machine, Swarm. In this blog, I will cover Docker Compose and how Docker Machine, Swarm and Compose can work with each other. The interworking part is actively being developed by Docker team and is still at the preliminary stages. Docker Compose: Docker Compose comes from Fig project. With Docker Compose, we can define a multi-container application in a YAML file along with the Container dependencies, affinities etc and Compose will take care of orchestrating the application. Following picture from Docker Compose presentation illustrates the point above. docker7 Following is a sample YAML file describing a small application with 2 containers, 1 for web and another for db. Continue reading Docker Compose and Interworking of Docker Machine, Swarm, Compose

Docker Swarm

This is a continuation of my previous blog on Docker machine. In this blog, I will cover Docker Swarm.

Swarm manages a set of Docker nodes as a single cluster. This has the following advantages:

  • Rather than managing individual Docker nodes, the cluster can be managed as a single entity.
  • Swarm has an in-built scheduler that will decide the placement of Containers in the cluster. There are different constraints and affinities that can be mentioned which Swarm uses to decide the Container placement. Constraints could be cpu, memory etc and Affinity could be for grouping related Containers together. Swarm also has the provision to take its scheduler out and work with other schedulers like Mesos and Kubernetes.
  • Swarm will take care of node failures so that Container HA can be provided.

Swarm has the following software components:

  • Swarm Manager that takes of scheduling and HA. HA piece is not yet available.
  • Swarm agent that runs in each node and communicates to Swarm manager.
  • Node discovery. There are different approaches available for Swarm worker nodes to discover Swarm master. Discovery is needed because Swarm master and agents run on different nodes and there is a need to find the discovery parameters dynamically. Available discovery mechanisms are docker hub, etcd, Consul etc.

Continue reading Docker Swarm

Docker Machine

As part of Docker Orchestration, Docker has released 3 new tools Machine, Swarm, Compose. In the last few weeks, I was playing with these tools and I will share my experiences in this blog. I will start with Docker machine in this blog and I will cover Swarm, Compose in the next set of blogs. Only preliminary version of these tools are released and there is a plan to release more updated versions later this year. For basics and other details on Docker, you can refer to my Docker blog series.

Docker Machine:

Docker machine makes it easier to create Docker hosts using an uniform approach across bare metal, VM, Cloud provider, Private clouds etc. Before Docker machine was there, following were the approaches available to create Docker hosts:

  • On Linux machines, Docker agent and client are installed natively.
  • For Windows, boot2docker is used to create a Docker host on top of hypervisor like Virtualbox.
  • For public clouds, we would create a Linux VM and install Docker on top of it.

Continue reading Docker Machine

Docker Orchestration

This blog is part of my ongoing series on Docker containers. Orchestrating Containers is a pretty complex task and there is a lot of work ongoing to solve this particular problem. There are big companies, startups as well as Opensource projects involved with this work. There are many different technologies and projects ongoing that got me really confused when I started looking at this. In this blog, I have tried to break down the Docker orchestration problem into smaller pieces and have tried to map different existing/developing solutions into the smaller pieces. Considering that the technologies are evolving and that my knowledge in this area is limited, this blog might need updates and corrections as we move forward. Also, I might have missed few technologies as well as companies..

Problem statement:

Docker does a great job in packaging and transporting single containers. Following are specific problems we need to address:

  • Distributed Applications split between multiple containers.
  • Manage a large number of containers both in terms of allocating the containers to the cluster of hosts as well as handling container failures.

Orchestration blocks:

Continue reading Docker Orchestration