Docker containers provides an isolated sandbox for the containerized program to execute. One-shot containers accomplishes a particular task and stops. Long running containers runs for an indefinite period till it either gets stopped by the user or when the root process inside container crashes. It is necessary to gracefully handle container’s death and to make sure that the Job running as container does not get impacted in an unexpected manner. When containers are run with Swarm orchestration, Swarm monitors the containers health, exit status and the entire lifecycle including upgrade and rollback. This will be a pretty long blog. I did not want to split it since it makes sense to look at this holistically. You can jump to specific sections by clicking on the links below if needed. In this blog, I will cover the following topics with examples:
Handling Signals and exit codes
Continue reading Docker features for handling Container’s death and resurrection
One of the big features in Docker 1.12 release is Swarm mode. Docker had Swarm available for Container orchestration from 1.6 release. Docker released Swarmkit as an opensource project for orchestrating distributed systems few weeks before Docker 1.12(RC) release. I had some confusion between these three projects. In this blog, I have tried to put my perspective on the similarities and differences between these three software components. I have also created a sample application and deployed it using the three approaches which makes it easier to compare.
Docker Swarm mode is fundamentally different from Swarm and it is confusing to use the same name Swarm. It would have been good if Docker could have renamed this to something different. Another point adding to the confusion is that native Swarm functionality will continue to be supported in Docker 1.12 release, this is done to preserve backward compatibility. In this blog, I have used the term “Swarm” to refer to traditional Swarm functionality, “SwarmNext” to refer to new Swarm mode added in 1.12, “Swarmkit” to refer to the plumbing open source orchestration project.
Swarm, SwarmNext and Swarmkit
Following table compares Swarm and SwarmNext:
Continue reading Comparing Swarm, Swarmkit and Swarm Mode
The most popular Container Orchestration solutions available in the market are Kubernetes, Swarm and Mesos. I have used Kubernetes and Swarm, but never got a chance to use Mesos or DC/OS. There were a bunch of questions I had about Mesos and DC/OS and I never got the time to explore that. Recently, I saw the announcement about Mesosphere opensourcing DC/OS and I found this as a perfect opportunity for me to try out Opensource DC/OS. In this blog, I have captured the answers to questions I had regarding Mesos and DC/OS. In the next blog, I will cover some hands-on that I did with Opensource DC/OS.
What is the relationship between Apache Mesos, Opensource DC/OS and Enterprise DC/OS?
Apache Mesos is the Opensource distributed orchestrator for Container as well as non-Container workloads. Both Opensource DC/OS and Enterprise DC/OS are built on top of Apache Mesos. Opensource DC/OS adds Service discovery, Universe package for different frameworks, CLI and GUI support for management and Volume support for persistent storage. Enterprise DC/OS adds enterprise features around security, performance, networking, compliance, monitoring and multi-team support that the Opensource DC/OS project does not include. Complete list of differences between Opensource and Enterprise DC/OS are captured here.
What does Mesosphere do and how it is related to Apache Mesos?
Mesosphere company has products that are built on top of Apache Mesos. Lot of folks working in Mesosphere contribute to both Apache Mesos and Opensource DC/OS. Mesosphere has the following products currently:
- DC/OS Enterprise – Orchestration solution
- Velocity- CI, CD solution
- Infinity – Big data solution
Why DC/OS is called OS?
Sometimes folks get confused thinking Mesos being a Container optimized OS like CoreOS, Atomic. Mesos is not a Container optimized OS. Similar to the way Desktop OS provides resource management in a single host, DC/OS provides cluster management across entire cluster. Mesos master(including first level scheduler) and agent are perceived as kernel components and user space components include frameworks, user space applications, dns and load balancers. The kernel provides primitives for the frameworks.
What are Mesos frameworks and why they are needed?
Continue reading Mesos and Mesosphere – FAQ
This is a continuation of my previous 2 blogs on Docker machine, Swarm. In this blog, I will cover Docker Compose and how Docker Machine, Swarm and Compose can work with each other. The interworking part is actively being developed by Docker team and is still at the preliminary stages. Docker Compose: Docker Compose comes from Fig project. With Docker Compose, we can define a multi-container application in a YAML file along with the Container dependencies, affinities etc and Compose will take care of orchestrating the application. Following picture from Docker Compose presentation illustrates the point above. Following is a sample YAML file describing a small application with 2 containers, 1 for web and another for db. Continue reading Docker Compose and Interworking of Docker Machine, Swarm, Compose
This is a continuation of my previous blog on Docker machine. In this blog, I will cover Docker Swarm.
Swarm manages a set of Docker nodes as a single cluster. This has the following advantages:
- Rather than managing individual Docker nodes, the cluster can be managed as a single entity.
- Swarm has an in-built scheduler that will decide the placement of Containers in the cluster. There are different constraints and affinities that can be mentioned which Swarm uses to decide the Container placement. Constraints could be cpu, memory etc and Affinity could be for grouping related Containers together. Swarm also has the provision to take its scheduler out and work with other schedulers like Mesos and Kubernetes.
- Swarm will take care of node failures so that Container HA can be provided.
Swarm has the following software components:
- Swarm Manager that takes of scheduling and HA. HA piece is not yet available.
- Swarm agent that runs in each node and communicates to Swarm manager.
- Node discovery. There are different approaches available for Swarm worker nodes to discover Swarm master. Discovery is needed because Swarm master and agents run on different nodes and there is a need to find the discovery parameters dynamically. Available discovery mechanisms are docker hub, etcd, Consul etc.
Continue reading Docker Swarm