Docker Machine

As part of Docker Orchestration, Docker has released 3 new tools Machine, Swarm, Compose. In the last few weeks, I was playing with these tools and I will share my experiences in this blog. I will start with Docker machine in this blog and I will cover Swarm, Compose in the next set of blogs. Only preliminary version of these tools are released and there is a plan to release more updated versions later this year. For basics and other details on Docker, you can refer to my Docker blog series.

Docker Machine:

Docker machine makes it easier to create Docker hosts using an uniform approach across bare metal, VM, Cloud provider, Private clouds etc. Before Docker machine was there, following were the approaches available to create Docker hosts:

  • On Linux machines, Docker agent and client are installed natively.
  • For Windows, boot2docker is used to create a Docker host on top of hypervisor like Virtualbox.
  • For public clouds, we would create a Linux VM and install Docker on top of it.

Docker machine provides a single command to create Docker hosts for all the cases mentioned above. Following picture from here is a good graphical presentation of what docker-machine can do:

docker4

Following are some things I tried:

Docker machine for Windows:
Virtualbox needs to be installed in the Windows machine since I have used docker-machine Virtualbox driver. Also, I installed msysgit for Windows as mentioned in the Docker machine blog, this gives a Unix feel-like environment.
I installed docker-machine version 0.2.0 using the procedure mentioned in Docker blog. I could create Docker hosts using:

docker-machine create --driver virtualbox dev

To use a particular Docker machine, we need to set the environment variable appropriately:

eval "$(docker-machine env dev)"

At this point, we can do a basic Container test:

docker run busybox echo hello world

Docker machine for Linux:

There are 2 issues I faced here:

  1. I was not able to run docker-machine inside Ubuntu running on Virtualbox within Windows. This is because Virtualbox does not support nested virtualization. I had to use VMWare player and then I was able to run docker-machine and create hosts.
  2. Virtualbox was hanging on “Waiting for VM to start”. This issue is mentioned in github machine issues and the workaround is to use docker-machine beta 0.3.0 version that can be downloaded using:
wget --no-check-certificate https://public.evanhazlett.com/docker-machine/vbox-intel-nic/docker-machine_linux-amd64  > docker-machine-0.3.0

Docker machine for Cloud providers:

docker-machine provides drivers to connect to Cloud providers like AWS, GCE, Azure etc. I tried docker-machine with Amazon AWS. I used the following command:

docker-machine create --driver=amazonec2 --amazonec2-access-key= --amazonec2-secret-key= --amazonec2-vpc-id= --amazonec2-region= awstest

It is important to pass accesskey, secretkey, vpcid and region. These details can be found from AWS console. After this, we can create containers on AWS like any other host. Just a sidenote: By default, docker-machine uses this linux aws ami “ami-898dd9b9” and this is not free-tier eligible.

Docker machine for native hosts:

For hosts where Docker is already installed, docker-machine provides a driver-less approach to connect to these hosts. I was not able to get this working. Following are the steps I tried:
Start docker on the linux host enabling TLS:

sudo /usr/bin/docker -d --tlsverify --tlscacert=/home/xxx/.docker/machine/certs/ca.pem --tlskey=/home/xxx/.docker/machine/certs/key.pem --tlscert=/home/xxx/.docker/machine/certs/cert.pem --host=unix:///var/run/docker.sock --host=tcp://0.0.0.0:2376

Connect from docker-machine:

docker-machine --tls-ca-cert=/home/xxx/.docker/machine/certs/ca.pem --tls-client-key=/home/xxx/.docker/machine/certs/key.pem --tls-client-cert=/home/xxx/.docker/machine/certs/cert.pem create --url=tcp://0.0.0.0:2376 custom3

I get this error when setting environment. I tried this with docker-machine on both Windows and Linux.

$ docker-machine env custom3
open /home/xxx/.docker/machine/machines/custom3/ca.pem: no such file or directory

I am discussing this issue on Machine github.

Native hosts(Update as of March 23, 2016):

I was able to get docker-machine working with native Linux hosts recently. I followed the procedure here, but I had to change it a little to get it working from my Windows machine. I tried 2 approaches and both of them worked fine:

  1. Get a Ubuntu Linux host where Docker is not installed.
  2. Have a Ubuntu Linux host where Docker is already installed. In this case, it is necessary for Docker client and Daemon to be of same version.

My environment:

  • Docker client and Docker-machine running from Windows.
  • Docker client version 1.10.3 and docker-machine version 0.5.6

Following are the summary of steps:

  • Create public, private key pair using “ssh-keygen -t rsa”. It is very important to not protect private key with password. docker-machine does not work with private key that is password protected.
  • Allow the public key using “ssh-copy-id” in Linux host.
  • Allow passwordless sudo access to Linux host using visudo by having this line “sudo   ALL=(ALL) NOPASSWD:ALL”
  • Copy private key to Windows host and run docker-machine. To verify that ssh works with the key, you can try “ssh -i <private key> <userid>@<host>”, this should work without password.

Following is the docker-machine that I executed and the relevant output. This output corresponds to the Linux host where Docker was not installed previously.

docker-machine create --driver generic  --generic-ip-address 172.17.8.111  --generic-ssh-user smakam14  --generic-ssh-key=/c/users/srmakam/.ssh/mydocker1  myubuntu1
Running pre-create checks...
Creating machine...
(myubuntu1) Importing SSH key...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Detecting the provisioner...
Provisioning with ubuntu(upstart)...
Installing Docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...

Following is the docker-machine output with machine successfully created:

$ docker-machine ls
NAME          ACTIVE   URL          STATE     URL                       SWARM   DOCKER    ERRORS
myubuntu1     -        generic      Running   tcp://172.17.8.111:2376           v1.10.3

To set the environment to the docker-machine from client, we can do:

eval $(docker-machine env --shell sh myubuntu1)

References:

Picture used in this blog are from references.

Advertisements

9 thoughts on “Docker Machine

  1. Just found your blog entry while looking for docker machine articles. Nice work.

    I have a similar issue about connecting docker-machine on already provisioned hosts. Did you finally solve the issue?

  2. Hi. I’m having trouble with this. I have an Ubuntu VM in MS Azure that was created in Azure and want to add it to the docker-machine list. I can create VMs from Docker-Machine (using the new driver commands > Docker Toolbox – Quick Terminal) no problem. I have created VMs in Azure with no ssh and with ssh. I’m trying to use the generic drive commands with no success.

    Error that is occurring is:

    (vm) No SSH key specified. Connecting to this machine now and in the future will require the ssh agent to contain the appropriate key.
    Waiting for machine to be running, this may take a few minutes…
    Detecting operating system of created instance…
    Waiting for SSH to be available…
    Error creating machine: Error detecting OS: Too many retries waiting for SSH to be available. Last error: Maximum number of retries (60) exceeded

    Any ideas?

    1. hi Derek
      Did you follow these steps as mentioned in the blog above? For first step, you might not need to generate a new keypair if the key pair is already generated for Azure ubuntu vm.

      Create public, private key pair using “ssh-keygen -t rsa”. It is very important to not protect private key with password. docker-machine does not work with private key that is password protected.
      Allow the public key using “ssh-copy-id” in Linux host.
      Allow passwordless sudo access to Linux host using visudo by having this line “sudo ALL=(ALL) NOPASSWD:ALL”
      Copy private key to Windows host and run docker-machine. To verify that ssh works with the key, you can try “ssh -i @”, this should work without password.

      Assuming above steps are done, docker-machine should work fine.

      Sreenivas

  3. Hey there!
    Thanks a lot for the article.
    I’ve managed to install docker on remote ubuntu 16 host from my mac.
    What i was missing and got from your article – no passphrase key. That was the problem.
    and i solved it. thatnk to your post. And you! Thanks

  4. The problem with this approach is that reprovisions docker keys (“create generic”) so older instances of docker-machine no longer work.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s