Docker Networking Tip – Macvlan driver

Last few months, I have been looking at Docker forums(https://forums.docker.com/, https://stackoverflow.com/questions/tagged/docker) and trying to understand some of the common questions/issues faced in the Docker Networking area. This prompted me to do 2 presentations:

I received positive feedback to these 2 presentations. As a next step, I thought preparing each Docker Networking tip as a video can help some folks to get a better picture. As a first attempt, I prepared Macvlan driver as my first Docker Networking video tip. Following is the associated Youtube video and presentation.

If you think this is useful and would like to see more videos, please let me know. Based on the feedback received, I will try to create more Docker Networking tips in video format.

Bangalore meetup – Docker Networking Troubleshooting presentation

Docker Bangalore meetup is a very active group dedicated to topics around Docker and the ecosystem around it. There was a meetup conducted yesterday at IBM office. There was a mix of topics presented including Moby, Linuxkit, Docker for Windows and Docker multi-stage builds. Thanks to Neependra for organizing the meetup, Neependra created this meetup group and has been passionately doing it from the early days of Docker. Thanks to IBM for hosting the meetup, IBM has been a very active contributor to Container and Docker ecosystem.

I did a presentation on “Docker Networking – Common issues and troubleshooting considerations”. The feedback received in the session was very positive. Following were some reasons that I did this presentation:

  • Networking is a complex topic and Docker networking is continuously evolving in every release and this makes it difficult for folks to figure out the right feature to use for their use-case.
  • When applications are taken from development to production, networking needs change and the typical approaches don’t always help.
  • Enterprise customers have lot of legacy applications and the networking needs to satisfy connecting legacy non-containerized applications with the new container based micro services.

Following is the approach and references I took to prepare the presentation:

  • Over the years, I have used some techniques for common networking stuff including Docker containers and I have captured this.
  • I went through the discussions in stackoverflow and docker forums and tried to find the common networking questions that folks ask.
  • Docker has a great documentation that includes Docker’s internal documentation, white papers and great set of blogs. I referred this extensively.
  • Dockercon videos around networking topics.

Following link has the slides and this link has the video recording to my presentation. The slides in the recording is not very clear. I would very much appreciate if folks can provide feedback and also provide suggestions on additional Docker networking tips that can be covered. I will try to keep the slides updated based on what I learn/understand in the future.

Following are some of my earlier blogs/presentations that you can refer to get more understanding on Docker Networking recent features:

Docker CE – Installing Test(“RC”) version

Starting with Docker 17.03, Docker introduced Community edition(CE) and Enterprise edition(CE) version of their Docker software. The release numbering also changed. From Docker 1.13.1, we jump to 17.03 version. Docker CE is the free version while Docker EE is the commercially supported enterprise version. Docker enterprise edition comes in different flavors based on the cost. Please refer this link for details on comparison between different Docker editions and the supported platforms for each editions. Both Docker CE and EE follow time based release schedule. Docker CE has 3 editions. CE “stable” edition gets released once every 3 months. CE “edge” edition gets released once every month. CE “test” edition is a release candidate that gets folded into “edge” and “stable” versions. I have used Docker release candidate(“test” edition) to try out new features before they get released. The steps to install release candidate Docker version is slightly different from installing “stable” and “edge” versions. Docker CE 17.06.0-ce-rc2 got released few days back and I have started trying out the new features in this version. This is a precursor to 17.06 release that will happen in few weeks. In this blog, I will cover installation steps for Docker CE edition release candidate software versions. I have focused on 17.06.0-ce-rc2, but the steps applies to any release candidate versions. The 3 approaches I have tried are installation from Docker static binaries, Docker machine with boot2docker and installation in Ubuntu platform with package manager.

Installation using Docker machine

When Docker RC version is released, the corresponding boot2docker image also gets released. I used the steps below to to the installation.

Continue reading Docker CE – Installing Test(“RC”) version

Compare Docker for Windows options

As part of Dockercon 2017, there was an announcement that Linux containers can run as hyperv container in Windows server. This announcement made me to take a deeper look  into Windows containers. I have worked mostly with Linux containers till now. In Windows, I have mostly used Docker machine or Toolbox till now. I recently tried out other methods to deploy containers in Windows. In this blog, I will cover different methods to run containers in Windows, technical internals on the methods and comparison between the methods. I have also covered Windows Docker base images and my experiences trying the different methods to run Docker containers in Windows. The 3 methods that I am covering are Docker Toolbox/Docker machine, Windows native containers, hyper-v containers.

Docker Toolbox

Docker Toolbox runs Docker engine on top of boot2docker VM image running in Virtualbox hypervisor. We can run Linux containers on top of the Docker engine. I have written few blogs(1, 2) about Docker Toolbox before. We can run Docker Toolbox on any Windows variants.

Windows Native containers

Continue reading Compare Docker for Windows options

Linux Docker base images

Recently, someone asked me how to create a Docker base image for a Linux variant that they are creating. In this blog, I will cover what a Linux base image is and how to create new base images.

Linux Docker base image

Following is a sample Dockerfile to create a Python webserver Docker container image.

FROM ubuntu:14.04

# Update the sources list
RUN apt-get update

# Update
RUN apt-get install -y python2.7 python-pip

# Install app dependencies
RUN pip install Flask==0.9.0

# Bundle app source
COPY simpleapp.py /src/simpleapp.py

EXPOSE  8000
CMD ["python", "/src/simpleapp.py", "-p 8000"]

In the above example, we have given “FROM ubuntu:14.04” in the first line. ubuntu:14.04 is the base image used here. 1 common question that I get asked is that does it mean that this container contains everything present in Ubunbu VM? That is not true. What ubuntu container image contains is the packages, libraries and tools associated with Ubuntu along with the root filesystem. To give a size comparison, Ubuntu container image is around 180mb while the Ubuntu VM size is around 1GB.

Docker hub contains base container images for all major distributions including Ubuntu, rhel, centos, debian etc. It is always better to take the official images as they would have the latest security patches. The official images are maintained by the distribution vendor who works closely with Docker.

Creating Linux Docker Container base image

Docker repository provides a script “mkimage.sh” that can be used to create base images for different Linux variants. When Docker is installed in Ubuntu 14.04, “mkimage.sh” is available in “/usr/share/docker-ce/contrib/”. Following output shows the “help” for “mkimage.sh”.

$ ./mkimage.sh --help
usage: mkimage.sh [-d dir] [-t tag] [--compression algo| --no-compression] script [script-args]
   ie: mkimage.sh -t someuser/debian debootstrap --variant=minbase jessie
       mkimage.sh -t someuser/ubuntu debootstrap --include=ubuntu-minimal --components=main,universe trusty
       mkimage.sh -t someuser/busybox busybox-static
       mkimage.sh -t someuser/centos:5 rinse --distribution centos-5
       mkimage.sh -t someuser/mageia:4 mageia-urpmi --version=4
       mkimage.sh -t someuser/mageia:4 mageia-urpmi --version=4 --mirror=http://somemirror/
       mkimage.sh -t someuser/solaris solaris

Each Linux distribution provides a helper script to create the base filesystem. For example, “debootstrap” is used for debian/ubuntu variants, “rinse” is used for centos variants. “mkimage.sh” uses the root filesystem created by helper script to import that to a Docker container. The helper script installs the necessary packages and sets up the root filesystem. If you are creating a new flavor of Linux and if it based on 1 of the major distributions, we can extend the helper script. Otherwise, we can create a new helper script based on the current examples. It will be good to contribute the new helper script back to “mkimage” in Docker repository.

Following is an example to create Debian Jessie base image with mkimage.sh:

sudo ./mkimage.sh -t smakam/debian:jessie debootstrap jessie

The above command will create a new container image “smakam/debian:jessie”. It pulls the necessary files from the repository.

References

Dockercon 2017 – My experiences

Dockercon 2017 was the first Docker global conference that I attended. The conference was hosted in Austin, Texas. It was a memorable experience and I had lot of fun attending the conference. In this blog, I will share some of my experiences from Dockercon 2017. I have covered details on important announcements, keynote demos, Cool hacks, Sessions that I attended, Security workshop conducted by me and Docker team and key takeaways for me.

Key announcements

Following were key announcements as part of Keynote sessions:

  • Moby opensource projectMoby is a framework to assemble specialized container systems. Docker is 1 of the assembled container systems from Moby. There can be other container systems that users can create. For example, 1 of the example demonstrated in keynote is to use Moby to build a container system to run Kubernetes on Mac. Moby is an effort to keep Docker open source projects and Docker product separate.
  • LinuxkitLinuxkit is a toolkit for building custom, minimal and immutable Linux distributions. This is used by Microsoft to run Linux containers in Windows. Linuxkit is 1 of the components of Moby that allows us to build a bootable container system. This system can be run either on bare-metal or on cloud.
  • IBM is running Docker in their powerpc and Z systems.
  • Oracle enterprise DB is available in Docker store and can be tried free for personal use.
  • I am glad to mention the Cisco announcements.  Cisco and Docker are partnering on Modernizing traditional applications(MTA) program. Contiv 1.0  is available as GA.

Keynote Demos

Live demos are a key part of Dockercon. These demos were done as part of keynote sessions from Solomon and Ben.

  • Multistage Docker build to reduce Docker image size and desktop to cloud integration for moving applications across Swarms.
  • Deploying an application securely with multiple services on Docker swarm cluster. The application was deployed with Docker compose using TLS, Secrets.
  • Secure supply chain using DDC, Security scan and Docker secrets
  • Deploying 3rd party VM applications with containers using image2docker and Docker datacenter. image2docker can do migration of VMs to Containers and this would be helpful for migrating legacy applications.

Hacks

Following 2 hacks were done by Docker captains. These were selected from the many hacks submitted for Dockercon.

  1. PWD – play with Docker
    PWD is a great tool for running Docker containers using browser without having to install Docker. This is great for workshops and it is also a good Docker beginner tool. For more details on PWD, please refer to my earlier blog here.
  2. FaaS – This is a framework for building serverless functions on Docker Swarm. The demo was a cool one with integration with Alexa service.

Sessions attended

Following are the sessions that I attended over the Dockercon week:

  • Cilium: Network and application security using BPF and XDP –
    • Berkeley packet filter(BPF) and extended data processing(XDP) runs in Linux kernel.
    • Learnt use cases of BPF where policy can be forced at network layer inside linux kernel using BPF.
    • Cilium can be used as Docker networking plugin.
    • XDP extends BPF to network drivers which makes packet filtering even faster. Facebook says that XDP is 10 times faster with switching packets.
  • Solving the storage problem for cloud-native applications – Portworx
    • Portworx is a container storage solution that is trying to solve the big problem of persistent container storage. This is a complex problem to solve and there are many players trying to address this problem.
  • Scaling App defense with intent based security – Twistlock
    • This session went into details of Twistlock container security platfrom. Dynamic secure policies can be created by Twistlock automatically.
  • Docker networking: from application plane to Data plane
    • Covered Docker networking from beginning to now including tools to debug common Docker networking issues.
  • Infinit’s next generation key value store
    • Covered how Infinit’s solution is unique, distributed and scalable. Object store and file system can be on top of key-value store, this is targeted for 4th quarter of this year.
  • Journey to Docker production: evolving your infrastructure and processes
    • Talk from Docker Captain Bret fisher – Explained the production considerations for small and big Docker clusters.
  • Container performance analysis – Netflix
    • Netflix tools to debug container performance. Covered tools like Netflix victor, titus, flame graphs.
  • From ARM to Z: multi-platform Docker swarm
    • Cross-platform containers using manifest tool. Same container image can be used across multiple platforms so that developer don’t need to remember platform details.
  • Building a secure app with Docker
    • Best practices to be followed for building secure applications

I am eagerly waiting to watch the recording of the other sessions.

Security workshop

I conducted Docker Security workshop along with Nigel, Nass, Matt. Nigel is a Docker captain and Nass and Matt are from Docker team. Around 50 folks attended the session. It was a 3 hr session with presentations and labs on different Docker security topics including Swarm mode, Content trust, Security scan, Networking, Secrets and Linux container security features. The labs were done on AWS cloud. The session was interactive and we got interesting questions from the audience. The labs and the slides are posted here.

What I enjoyed the most

  • Meeting the folks in person with whom I have interacted over mails and slack
  • Keynote demos
  • Interacting with other Docker captains. Docker has an amazing Captains group and I am privileged to be part of the group. From Bangalore, 2 other Docker captains Neependra and Ajeet also attended the conference. Following picture was taken in the Captains summit.

captains_picture

  • Captains discussion with Solomon Hykes.
  • Presenting and interacting with folks in Docker Security workshop
  • Seeing overall Docker excitement with attendees
  • Talking to companies in their booth and understanding container ecosystem
  • Everyday after conference party…

 

 

Comparing Docker deployment options in public cloud

Few weeks back, I gave a presentation in Container conference, Bangalore comparing different solutions available to deploy Docker in the public cloud.

Slides are available here. I have also put the steps necessary along with short video for each of the options in the github page here.

Abstract of the talk:

Containers provide portability for applications across private and public clouds. Since there are many options to deploy Docker Containers in public cloud, customers get confused in the decision making process. I will compare Docker machine, Docker Cloud, Docker datacenter, Docker for AWS, Azure and Google cloud, AWS ECS, Google Container engine, Azure Container service. A sample multi-container application will be deployed using the different options. The deployment differences including technical internals for each option will be covered. At the end of the session, the user will be able to choose the right Docker deployment option for their use-case.

Note:

  • I have focused mainly on Docker centric options in the comparison.
  • There are few CaaS platforms like Tectonic, Rancher that I have not included since I did not get a chance to try them.
  • Since all the solutions are under active development, some of the gaps will get covered by the solutions in the future.