Category Archives: boot2docker

Compare Docker for Windows options

As part of Dockercon 2017, there was an announcement that Linux containers can run as hyperv container in Windows server. This announcement made me to take a deeper look  into Windows containers. I have worked mostly with Linux containers till now. In Windows, I have mostly used Docker machine or Toolbox till now. I recently tried out other methods to deploy containers in Windows. In this blog, I will cover different methods to run containers in Windows, technical internals on the methods and comparison between the methods. I have also covered Windows Docker base images and my experiences trying the different methods to run Docker containers in Windows. The 3 methods that I am covering are Docker Toolbox/Docker machine, Windows native containers, hyper-v containers.

Docker Toolbox

Docker Toolbox runs Docker engine on top of boot2docker VM image running in Virtualbox hypervisor. We can run Linux containers on top of the Docker engine. I have written few blogs(1, 2) about Docker Toolbox before. We can run Docker Toolbox on any Windows variants.

Windows Native containers

Folks familiar with Linux containers know that Linux containers uses Linux kernel features like namespaces, cgroups. To containerize Windows applications, Docker engine for Windows needs to use the corresponding Windows kernel features. Microsoft worked with Docker to make this happen. As part of this effort, changes were made both on Docker and Windows side. This mode allows Windows containers to run directly on Windows server 2016. Windows server 2016 has the necessary container primitives that allows native Windows containers to run on it. Going forward, Microsoft will port this functionality to other flavors of Windows.

hyper-v containers

Windows Hyper-v container is a windows server container that runs in a VM. Every hyper-v container creates its own VM. This means that there is no kernel sharing between the different hyper-v containers. This is useful for cases where additional level of isolation is needed by customers who don’t like the traditional kernel sharing done by containers. The same Docker image and CLI can be used to manage hyper-v containers. Creation of hyper-v containers is specified as a runtime option. There is no difference when building or managing containers between windows server and hyper-v container. Startup times for hyper-v container is higher than windows native container since a new lightweight VM gets created each time. 1 common question that comes up is how is hyper-v container different from a general VM with virtualbox or hyper-v hypervisor and running container on top of it? Following are some differences as I see it:

  • hyper-v container is very light-weight.  This is because of the light-weight OS and other optimizations.
  • hyper-v containers do not appear as VMs inside hyper-v and cannot be managed by regular hyper-v tools.
  • The same Docker  CLI can be used to manage hyper-v containers. To some extent, this is true with Docker Toolbox and Docker machine. With hyper-v containers, its more integrated and becomes a single step process.

There are 2 modes of hyper-v container.

  1. Windows hyper-v container – Here, hyper-v container runs on top of Windows kernel. Only Windows containers can be run in this mode.
  2. Linux hyper-v container – Here, hyper-v container runs on top of Linux kernel. This mode was not available earlier and it was introduced as part of Dockercon 2017. Any Linux flavor can be used as the base kernel. Docker’s Linuxkit project can be used to build the Linux kernel needed for the hyper-v container. Only Linux containers can be run in this mode.

We cannot use Docker Toolbox and hyper-v containers at the same time. Virtualbox cannot run when “Docker for Windows” is installed.

Following picture shows illustration of different Windows container modes


Following table captures the difference between different Windows Container modes

Windows mode/Feature Toolbox Windows native container hyper-v container
OS Type Any Windows flavor Windows 2016 server Windows 10 pro, Windows 2016 server
hypervisor/VM Virtualbox hypervisor No seperate VM for container VM runs inside hyper-v
Windows container Not possible Yes Possible in Windows hyper-v container
Linux container Yes Not possible Possible in Linux hyper-v container
Startup time Higher compared to windows native and hyper-v containers Least among the 3 options Between Toolbox and windows native containers


If you are using Windows 10 pro or Windows server 2016, you can install Docker for Windows from here. This installs Docker CE version and runs Docker for Windows in hyper-v mode. We can install using either the stable or edge channel. Docker for Windows was available earlier only for Windows 10. The edge channel added Docker for Windows for Windows server 2016 just recently. Once “Docker for Windows” is installed, we can switch between Linux and Windows mode with just a click of a button. As of now, Linux mode uses mobyLinuxVM, this will change later to hyper-v linux container mode. In order to run Hyper-V containers, the Hyper-V role has to be enabled in Windows. If the Windows host is itself a Hyper-V virtual machine, nested virtualization will need to be enabled before installing the Hyper-V role. For more details, please refer these 2 references(1, 2). As shown in the example of reference, we can start hyper-v containers by just specifying a run-time option in Docker.

docker run -it --isolation=hyperv microsoft/nanoserver cmd

If you are using Windows server 2016, Docker EE edition can be installed using the procedure here. I would advise using Docker EE for Windows server 2016 rather than using hyper-v container.

I have tried Docker Toolbox in Windows 7 Enterprise version. Docker Toolbox can be run in any version of Windows. Docker Toolbox installation also installs Virtualbox if its not already installed. Docker Toolbox can be installed from here. For Docker Toolbox hands-on example, please refer to my earlier blog here.

I tried out Windows native containers and hyper-v containers in Azure cloud. After I created a Windows 2016 server, I used the following commands to install Docker engine. These commands have to be executed from powershell in administrator mode.

Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
Install-Package -Name docker -ProviderName DockerMsftProvider
Restart-Computer -Force

Following are some example Windows containers I tried:

docker run microsoft/dotnet-samples:dotnetapp-nanoserver
docker run -d --name myIIS -p 80:80 microsoft/iis

Since Azure uses hypervisor to host compute VM and the fact that nested virtualization is not supported in Azure, Docker for Windows cannot be used with Windows server 2016 in Azure.
I got following error when I started “Docker for Windows” in Linux mode.

Unable to write to the database. Exit code: 1
   at Docker.Backend.ContainerEngine.Linux.DoStart(Settings settings) in C:\gopath\src\\docker\pinata\win\src\Docker.Backend\ContainerEngine\Linux.cs:line 243
   at Docker.Backend.ContainerEngine.Linux.Start(Settings settings) in C:\gopath\src\\docker\pinata\win\src\Docker.Backend\ContainerEngine\Linux.cs:line 120
   at Docker.Core.Pipe.NamedPipeServer.<>c__DisplayClass8_0.b__0(Object[] parameters) in C:\gopath\src\\docker\pinata\win\src\Docker.Core\pipe\NamedPipeServer.cs:line 44
   at Docker.Core.Pipe.NamedPipeServer.RunAction(String action, Object[] parameters) in C:\gopath\src\\docker\pinata\win\src\Docker.Core\pipe\NamedPipeServer.cs:line 140

I was still able to use hyper-v containers in Azure in Windows mode in Windows server 2016. I am still not fully clear how this mode overcame the nested virtualization problem.

From Azure perspective, I would like to see these changes from Microsoft:

  • Azure supporting nested virtualization.
  • Allowing Windows 10 in Azure without MSDN subscription.

There was an announcement earlier this week at Microsoft Build conference that Azure will support nested virtualization in selected VM sizes. This is very good.

Windows base image

Every container has a base image that contains the needed packages and libraries. Windows containers supports 2 base images:

  1. microsoft/windowsservercore – a full blown Windows server with full .NET Framework support. The size is around 9 GB.
  2. microsoft/nanoserver – a minimal Windows server and .NET Core Framework. The size is around 600 MB.

Following picture from here shows the compatibility between Windows server OS, Container type and Container base image.


As we can see from the picture, with hyper-v container, we can use only nanoserver container base image.


Can I run Linux containers in Windows?

  • The answer depends on which Docker windows mode you are using. With Toolbox and hyper-v Linux containers, Linux containers can be run in Windows. With Windows native container mode, Linux containers cannot be run in Windows.

Which Docker for Windows mode should I use?

  • For development purposes, if there is a need to use both Windows and Linux containers, hyper-v container can be used. For production purposes, we should use Windows native container. If there is a need to have better kernel isolation for additional security, hyper-v container can be used. If you have a version of Windows that is neither Windows 10 or Windows server 2016, Docker Toolbox is the only option available.

Can we run Swarm mode and Overlay network with Windows containers?

  • Swarm mode support was added recently in Windows containers. Multiple containers across Windows hosts can talk over the Overlay network. This needs Windows server update as mentioned in the link here. The same link also talks about a mixed mode Swarm cluster with Windows and Linux nodes. We can have a mix of Windows and Linux containers talking to each other over the Swarm cluster. Using Swarm constraints scheduling feature, we can place Windows containers in Windows nodes and Linux containers in Linux nodes.

Is there an additional Docker EE license needed for Windows server 2016?

  • According to the article here, it is not needed. It is better to check as this might change. Obviously, Windows license has to be taken care separately.


Service Discovery and Load balancing Internals in Docker 1.12

Docker 1.12 release has revamped its support for Service Discovery and Load balancing. Prior to 1.12 release, support for Service discovery and Load balancing was pretty primitive in Docker. In this blog, I have covered the internals of Service Discovery and Load balancing in Docker release 1.12. I will cover DNS based load balancing, VIP based load balancing and Routing mesh.

Technology used

Docker service discovery and load balancing uses iptables and ipvs features of Linux kernel. iptables is a packet filtering technology available in Linux kernel. iptables can be used to classify, modify and take decisions based on the packet content. ipvs is a transport level load balancer available in the Linux kernel.

Sample application

Following is the sample application used in this blog:

Continue reading Service Discovery and Load balancing Internals in Docker 1.12

Experimental Docker with Docker machine

Docker Experimental channel is used to release experimental Docker features so that Docker users can try the new features and provide feedback.  It is nice to use the experimental Docker in a test environment rather than upgrading Docker in the main development machine. The preferred approach is to use docker-machine and create a VM with experimental Docker. In this blog, I will describe the approach that I use to create docker-machine with experimental Docker VM. For basics of Docker machine, please refer to my blog on Docker machine.

Following are the steps needed to build the experimental boot2docker ISO and copy it to the docker-machine default location:

git clone
cd boot2docker
docker build -t my-boot2docker-img -f Dockerfile.experimental .
docker run --rm my-boot2docker-img > boot2docker.iso
mv boot2docker.iso ~/.docker/machine/cache/boot2docker.iso

We need to specify Docker experimental release location in Dockerfile.experimental. In this case, it is

Following command will start a Docker machine in Virtualbox with experimental Docker:

docker-machine create -d virtualbox exp

Following is the experimental Docker version running in my host:

$ docker --version
Docker version 1.11.0-dev, build 6c2f438, experimental

Installing custom software in boot2Docker image:
I had a recent usecase where I needed boot2docker to have ipvsadm installed. Package manager is not available in boot2docker. Other than installing ipvsadm, I had to copy few libraries. Following is my boot2docker.experimental file that I used for this usecase:

FROM boot2docker/boot2docker
MAINTAINER Sreenivas Makam ""

#DESCRIPTION use the latest experimental build of Docker

RUN apt-get update && apt-get install -y ipvsadm
RUN cp /sbin/ipvsadm $ROOTFS/sbin/
RUN cp /lib/x86_64-linux-gnu/ /rootfs/lib/
RUN cp /lib/x86_64-linux-gnu/ /rootfs/lib/
RUN cp /lib/x86_64-linux-gnu/ /rootfs/lib/

#get the latest experimental docker
RUN cd $ROOTFS/usr/local/bin && curl -fL -O && tar -xvzf docker-1.12.0-rc4.tgz --strip-components=1 && chmod +x $ROOTFS/usr/local/bin/docker* && rm docker-1.12.0-rc4.tgz

RUN echo "" >> $ROOTFS/etc/motd
RUN echo "  WARNING: this is an build, not a release." >> $ROOTFS/etc/motd
RUN echo "" >> $ROOTFS/etc/motd

CMD ["cat", "boot2docker.iso"]

Issue faced:
I was not able to use custom Docker image with docker-machine version 0.8.0-rc1. I could not find an option to prevent docker-machine from downloading latest Docker image. I have opened an issue here. The only workaround I found was to copy boot2docker image to ~/.docker/machine/cache/ , remove internet connection and then create docker-machine host.


Docker Toolbox

Docker Toolbox simplifies the creation of Docker environment for Windows and Mac. This deprecates boot2docker. Following components are included in Docker Toolbox.

  • Docker Client
  • Docker Machine
  • Docker Compose (Mac only)
  • Docker Kitematic
  • VirtualBox

I recently tried out Docker Toolbox. I had few issues to get it working and after some hiccups, I was able to get it working with some help from mailers. In this blog, I will share my experiences.

Continue reading Docker Toolbox

Docker Kitematic for Windows

Kitematic simplifies Docker installation for Mac and also provides an easy GUI interface to manage Containers. Recently, Kitematic released an alpha version for Windows. I tried this recently and I will share my experiences in this blog.

I had covered boot2docker in 1 of my earlier blog. Kitematic is boot2docker combined with a GUI for managing containers. Kitematic uses the same Linux VM as boot2docker.


After registering for Alpha program, I received the download link 1 day later. Kitematic loads a small Linux VM in Virtualbox over which the Container runs. If Virtualbox is not installed, Kitematic installation program also installs Virtualbox. In my case, I already had Virtualbox installed. When I tried installing, I got this issue:


I had faced similar issue when installing boot2docker. 1 of the workarounds I found by googling was to delete the host-only interface that Virtualbox creates for Kitematic/boot2docker. I already had 6 host-only interfaces created in my Virtualbox. I deleted 4 of them and tried reinstalling Kitematic and the installation went through after that.

The second issue I faced was with running Docker CLI and it complained that Docker client and agent versions did not match.This was because I had boot2docker installed previously in my Windows machine. I uninstalled boot2docker and Docker CLI worked fine after that.

Kitematic Networking:

Continue reading Docker Kitematic for Windows

Docker hands-on with boot2docker

This blog is part of my Docker series. In my previous blog, I covered installation and usage of Docker on Ubuntu Linux. I had installed Ubuntu Linux as a VM inside Virtualbox running in Windows 7. For folks who are interested in trying Docker using a simpler approach on Windows or Mac, boot2docker is an option.


Docker is supported currently only on Linux kernel. boot2docker is a tiny distribution of Linux on top of which Docker is installed. Installing boot2docker on Windows would allow us to run Docker on Windows machine. Actually boot2docker runs as a VM on top of Virtualbox. During installation, an option is given to install Virtualbox along with boot2docker. If you have Virtualbox already installed, you can select installation of only boot2docker, otherwise you can install both together. boot2docker for Windows can be installed from here. It does not make much sense to run boot2docker in Linux since Docker is supported natively in Linux and there will be performance impact of running boot2docker in a VM under Linux. Infact, I did not seen an option to install boot2docker under Linux.

Continue reading Docker hands-on with boot2docker