Flocker – Docker Data volumes

In this blog, I will cover Flocker from ClusterHQ which is used for Container data/volume management. I will cover some Flocker basics and some hands-on stuff that I tried with Flocker.

Docker Volumes:

Docker volumes are used to store persistent data that is outside the scope of the Container Union file system. 1 common example is database. Data volumes are attached to the container so that Containers can update the volume and these volumes can be reattached to other containers or shared across Containers. Data volumes are not deleted when Containers are deleted, Data volumes needs to be deleted manually.

Following is a simple example to create container with mount volume. Here, /webapp is mounted as a data volume.

docker run -d -P --name web -v /webapp training/webapp python app.py

When Containers are deleted, volumes don’t get deleted. To delete volumes when Containers are deleted, we can use:

docker rm -v

Volumes are stored in host under /var/lib/docker/volumes.

Flocker basics:

Docker Data volumes are tied to a single node where Container is created. When Containers are moved across nodes, Data volumes don’t get moved. Flocker mainly addresses this issue of moving the Data volumes along with the Container.

Following picture from ClusterHQ illustrates the above point.


Following picture from ClusterHQ describes the architecture of Flocker.


  • Flocker agent runs in each node and it takes care of talking to the Docker daemon and Flocker control service.
  • Flocker control service takes care of managing the volumes as well as managing Flocker cluster.
  • Currently supported storage includes Amazon AWS EBS, Rackspace block storage, EMC ScaleIO. Local storage is available on experimental basis.
  • Both REST api and Flocker CLI is used to manage volumes as well as Docker containers.

Flocker as Docker plugin:

In the current released version of Flocker, it not possible to manage Docker volumes using Flocker with Docker frontend tools. With Docker and Flocker experimental release, it is possible for Docker to manage volumes using Flocker as a Data volume plugin. Following picture from ClusterHQ illustrates the Docker with Flocker plugin architecture.


  • Data volumes will be managed using Docker front-end tools. This means all Docker Orchestration tools like Compose, Swarm can manage Docker volumes.
  • Docker will support multiple plugins to manage Data volumes and Flocker will be 1 of the plugins.
  • Flocker plugin will take care of managing data volumes and this includes migrating the volume associated with the Container when Container moves across hosts.
  • Flocker will use the Container networking technology to talk across hosts, this can be native Docker networking or Docker networking plugins.

Trying out Flocker:

I tried the following approaches to get some hands-on with Flocker.

Approach 1(Using VMs provided by ClusterHQ):

ClusterHQ provides Cloud based VMs that illustrates the basic Volume movement associated with Containers. To sign up, register here. I tried this and it worked pretty well. After signing up, we get an email with access details and we are given 3 hr window to use the Cloud VMs. We can try this multiple times if we cannot finish it the first time. There are 2 samples provided.

web and redis container are initially running on same host. We create some data on redis and then move redis container to different host. Redis data gets preserved after the Container gets moved.
Create mongodb container on host1, add some entries to db, move the db to different host and db entries would still be preserved.

Approach 2(Using Vagrant cluster):

  • I had a Ubuntu 14.04 VM with Virtualbox and Vagrant already installed.
  • Installed flocker-cli using the procedure here.

I followed the tutorial described here. The initial steps here uses Vagrant to create a 2 node Flocker cluster with 1 of the nodes running Flocker control service as well. The tutorial covers 3 examples:

  1. Mongo db container application getting moved across hosts with no data preserving.
  2. Mongo db container with port forwarding application getting moved across hosts with no data preserving.
  3. Mongo db container with port forwarding application with data volumes getting preserved when Container moves across hosts.

I was able to get examples 1 and 2 working fine. Example 3 did not work for me. When I created Mongo db entries with Container running in node 1 and when I moved the Container to node2, I did not see the data getting preserved. I have posted my issue in the ClusterHQ discussion forum here.

Approach 3(Experimental Flocker install with AWS):

  • I used Ubuntu 14.04 VM.
  • I used the procedure here to install Flocker experimental tools as well as to setup 2 AWS node cluster.

Initially, I did not install Flocker Docker plugin and I tried the steps in the tutorial to create a dataset. The default dataset creation in the tutorial asks for 100GB,  I tried to create 100MB dataset considering AWS space constraints with free-tier, the dataset was stuck in pending state.

curl -s --cacert $PWD/cluster.crt --cert $PWD/root.crt --key $PWD/root.key --header "Content-type: application/json" -XPOST -d '{"primary": "'${NODE_UUID}'", "maximum_size": 104857600, "metadata": {"name": "mongodb_data"}}' https://52.10.xx.xxx:4523/v1/configuration/datasets| jq .

$ flocker-volumes list
6457b725-462c-445e-89cc-62414bb704b8 0.10G name=mongodb_data pending ⌛ ccdd8495 (

I have highlighted my issue in the discussion forum here and waiting for response. I tried installing Docker plugin as described here but I could not create a container because the default volume size used is 100GB and we currently dont have an option to change this. This issue is already tracked here.

Approach 4(Docker with Flocker and Weave plugin):

Following link captures the demo done at Dockercon and this demonstrates Docker integration with Weave and Flocker plugin. The demo shows an application where redis container is moved between hosts using Weave networking and Flocker takes care of preserving Data volume. The detailed steps to run the demo locally is mentioned here. I followed the steps and I got stuck in 1 of the intermediate steps. I have posted my query here and still waiting for response.


Container data volume management is an important piece and Flocker seems to be covering all sides. Its an important step to have Flocker as a data volume plugin as this will integrate Flocker into the Docker mainstream tools. Its not clear how comprehensive Docker’s native volume management solution will be. For networking, Docker has a native batteries-included approach which can be replaced by Networking plugins like Weave, its not clear how this will evolve for Data volumes. I had multiple issues while trying out Flocker and I have mentioned this in the blog. I guess this is expected since its in the early stages. I will update this blog in case I am able to resolve the issues mentioned in this blog. If anyone has solved these issues, please let me know.

Solutions to some of the issues mentioned in the blog can be found here. Thanks to Flocker team.


Pictures used in this blog are from references.

4 thoughts on “Flocker – Docker Data volumes

  1. Sreeni, I am writing a Flocker backend driver for a hybrid storage device. I want to debug the backend driver code. ClusterHQ suggests to use Python module ‘eliot’, but I could not get it working as the documentation is not sufficient for me to start using it.
    I would greatly appreciate if you could help me on how to debug a Flocker backend driver. You can reach me on my email: sayasree AT gmail DOT com,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s