Docker Networking – Weave

This blog is part of my ongoing series on Docker containers. Weaveworks is developing a Docker Networking solution to connect Containers. I recently played around with their solution and in this blog, I will capture some of my thoughts.

Following are some internals on their implementation as I understood:

  • Weave creates a Weave bridge as well as a Weave router in the host machine.
  • Weave router establishes both tcp and udp connection across hosts to other Weave routers. TCP connection is used for discovery and protocol related exchange. UDP is used for data encapsulation. Encryption can be done if needed.
  • The Weave bridge is configured to sniff the packets that needs to be sent across hosts and redirect to the Weave router. For local switching, weave router is not used.

Following is a picture of the data path between 2 containers on 2 different hosts.

docker_net4

My environment:

I have 2 hosts running Ubuntu 14.04 in Virtualbox environment. The VMs have connectivity over Host-only adapter.

Demo:

Use the procedure here to install Weave.

I executed the following on host 1:

$ sudo weave launch
1ffabddc2250ee3a23889d6a92582cd555649c71f6683df81f75eab558f4b5cf
smakam14@sreeubuntu14-VirtualBox:~$ sudo docker ps
CONTAINER ID        IMAGE                COMMAND                CREATED             STATUS              PORTS                                            NAMES
1ffabddc2250        zettio/weave:0.8.0   "/home/weave/weaver    4 seconds ago       Up 3 seconds        0.0.0.0:6783->6783/tcp, 0.0.0.0:6783->6783/udp   weave               

I executed the following on host 2:

$ sudo weave launch 192.168.56.101
d86f383046c0e61a6864fe5f1598183544dc8be65e7ae109cdbf1a28ca369318
$ sudo docker ps
CONTAINER ID        IMAGE                COMMAND                CREATED              STATUS              PORTS                                            NAMES
d86f383046c0        zettio/weave:0.8.0   "/home/weave/weaver    About a minute ago   Up About a minute   0.0.0.0:6783->6783/tcp, 0.0.0.0:6783->6783/udp   weave         

In the second host, we need to specify the neighboring host ip address to which we are connecting to.

Following command executed on host 1 shows the weave status.

$ sudo weave status
[sudo] password for smakam14: 
weave router 0.8.0
Our name is 7a:04:87:a1:cc:8b
Sniffing traffic on &{30 65535 ethwe 8a:61:80:9f:6e:2d up|broadcast|multicast}
MACs:
82:27:03:4c:1a:10 -> 7a:04:87:a1:cc:8b (2015-01-12 08:08:35.073513253 +0000 UTC)
Peers:
Peer 7a:04:87:a1:cc:8b (v3) (UID 17978700228787792823)
   -> 7a:ae:3e:dc:91:56 [192.168.56.102:51597]
Peer 7a:ae:3e:dc:91:56 (v1) (UID 17406575075286523660)
   -> 7a:04:87:a1:cc:8b [192.168.56.101:6783]
Routes:
unicast:
7a:04:87:a1:cc:8b -> 00:00:00:00:00:00
7a:ae:3e:dc:91:56 -> 7a:ae:3e:dc:91:56
broadcast:
7a:04:87:a1:cc:8b -> [7a:ae:3e:dc:91:56]
7a:ae:3e:dc:91:56 -> []
Reconnects:

The above output shows the following:

  • Hostname which is the mac address of the Weave router.
  • Connection established to the peer weave router.

Following command shows the tcp and udp connections from netstat command on host 1:

udp6       0      0 [::]:6783               [::]:*
tcp6       0      0 [::]:6783               [::]:*                  LISTEN

Following is the bridge output command output:

$ sudo brctl show
bridge name	bridge id		STP enabled	interfaces
docker0		8000.56847afe9799	no		vethbbac2bd
weave		8000.7a0487a1cc8b	no		vethwepl5971

The above output shows the 2 bridges, docker0 and weave.

Now, lets create a container on host 1 and host 2:

host 1:

$ C=$(sudo weave run 10.0.1.1/24 -t -i ubuntu)
smakam14@sreeubuntu14-VirtualBox:~$ sudo docker ps
CONTAINER ID        IMAGE                COMMAND                CREATED             STATUS              PORTS                                            NAMES
5276bb4a689c        ubuntu:latest        "/bin/bash"            5 seconds ago       Up 4 seconds                                                         romantic_wright     
1ffabddc2250        zettio/weave:0.8.0   "/home/weave/weaver    About an hour ago   Up About an hour    0.0.0.0:6783->6783/tcp, 0.0.0.0:6783->6783/udp   weave    

host 2:

$ C=$(sudo weave run 10.0.1.2/24 -t -i ubuntu)
smakam14@sreeubuntu14-VirtualBox-1:~$ sudo docker ps
CONTAINER ID        IMAGE                COMMAND                CREATED             STATUS              PORTS                                            NAMES
9101765997ac        ubuntu:14.04         "/bin/bash"            3 seconds ago       Up 3 seconds                                                         thirsty_albattani   
b1720460998b        zettio/weave:0.8.0   "/home/weave/weaver    About an hour ago   Up About an hour    0.0.0.0:6783->6783/tcp, 0.0.0.0:6783->6783/udp   weave  

Following is the bridge table output in host 1:

$ sudo brctl show
bridge name	bridge id		STP enabled	interfaces
docker0		8000.56847afe9799	no		veth2d36381
							vethbbac2bd
weave		8000.7a0487a1cc8b	no		vethwepl5971
							vethwepl6407

Following is the ifconfig output in container 1:

# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:ac:11:00:07  
          inet addr:172.17.0.7  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:acff:fe11:7/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:648 (648.0 B)  TX bytes:648 (648.0 B)

ethwe     Link encap:Ethernet  HWaddr ae:11:29:f6:d7:db  
          inet addr:10.0.1.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::ac11:29ff:fef6:d7db/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:65535  Metric:1
          RX packets:36 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:5196 (5.1 KB)  TX bytes:648 (648.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

“ethwe” interface in the container is connected to “vethwepl6407” interface in weave bridge.

Lets try to ping the container1 to container2 and it is successful!

# ping -c1 10.0.1.2
PING 10.0.1.2 (10.0.1.2) 56(84) bytes of data.
64 bytes from 10.0.1.2: icmp_seq=1 ttl=64 time=44.8 ms

--- 10.0.1.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 44.832/44.832/44.832/0.000 ms

Following are some things that I am not sure of:

  • Its not clear about the purpose of “Docker” bridge and “eth0” interface created in the container. The container seems to create an interface towards the “Docker” bridge. When I shut the Docker bridge, the ping did not work. So I assume “Docker” bridge is playing a role in addition to the “Weave” bridge.
  • I am not sure how much extra overhead the sniffing the packets and encapsulating in UDP creates. Will this cause decrease in packet throughput?
  • I could not find a way to do multi-tenant networks.

Weave has some interesting capabilities like Multi-hop routing and encryption capabilities. Thanks to Weaveworks team for the nice solution.

References:

Advertisements

One thought on “Docker Networking – Weave

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s