Comparing Docker compose versions

In this blog, I have captured some of my learnings on Docker compose files and how they differ between versions. Docker compose is a tool used for defining and running multi-container Docker applications. I have used the famous multi-container voting application to illustrate the differences with compose versions.

Following are some questions that I have to tried to answer in this blog:

  • What is the difference between Compose versions 1, 2 and 3?
  • What is the difference between compose, stack and dab formats?
  • What are different ways to run compose files with different compose versions?
  • How does “docker stack deploy” really work?

Compose versions:

Following table captures the main differences between Compose versions:

Feature Compose v1 Compose v2 Compose v3
Docker engine version 1.9.1+ 1.10.0+ 1.13.0+
Compose version Compose up to 1.6.x Compose 1.6.0+ Compose 1.10.0+
Connect containers Bridge network, need links Overlay supported, networks can be used to connect containers Overlay supported, networks can be used to connect containers
Volume No named volume Named volume supported Named volume supported
Swarm capability Older swarm Older swarm Swarm mode
Deploy section No deploy section No deploy section Deploy section with replica present

There could be many reasons why someone would want to use older compose versions. It could either be because they are still using older Docker versions or they already have compose files that are currently in use. I have captured the famous votingapp in all 3 formats to capture the differences:

votingapp in compose v1:

Following is the votingapp in compose v1 format. This establishes container connectivity using links and can be deployed in standalone node or in cluster using old swarm mode.

vote:
    image: docker/example-voting-app-vote:latest
    environment:
      - "constraint:node==swarm-agent-753F9D8C000000"
    ports:
      - "5000:80"
    links:
      - redis

redis:
    image: redis:alpine
    environment:
      - "constraint:node==swarm-agent-753F9D8C000000"
    ports: ["6379"]

worker:
    image: docker/example-voting-app-worker:latest
    environment:
      - "constraint:node==swarm-agent-753F9D8C000000"
    links:
      - redis
      - db

db:
    image: postgres:9.4
    ports: ["5432"]
    environment:
      - "constraint:node==swarm-agent-753F9D8C000000"

result:
    image: tmadams333/example-voting-app-result:latest
    ports:
      - "5001:80"
    environment:
      - "constraint:node==swarm-agent-753F9D8C000000"
    links:
      - db

Following command can be used to deploy:

docker-compose up -d

votingapp in compose v2:

Following is the votingapp in compose v2 format. This establishes container connectivity using networks and can be deployed in standalone node or in cluster using old swarm mode. Containers connected through networks can be deployed in separate nodes and they can talk through overlay network here.

version: "2"

services:
  vote:
    image: docker/example-voting-app-vote:latest
    labels:
     - "com.example.description=Vote"
    ports:
      - "5000:80"
    networks:
      - front-tier
      - back-tier

  redis:
    image: redis:alpine
    ports: ["6379"]
    networks:
      - back-tier

  worker:
    image: docker/example-voting-app-worker:latest
    networks:
      - back-tier

  db:
    image: postgres:9.4
    ports: ["5432"]
    labels:
     - "com.example.description=Postgres Database"
    networks:
      - back-tier

  result:
    image: tmadams333/example-voting-app-result:latest
    ports:
      - "5001:80"
    networks:
      - front-tier
      - back-tier

networks:
  front-tier:
  back-tier:

Following command can be used to deploy:

docker-compose up -d

votingapp in compose v3:

Following is the votingapp in compose v3 format. This establishes container connectivity using networks and can be deployed in standalone node or in new swarm mode. This compose format has the deploy section that controls deployment parameters like number of replicas, constraints etc.

version: "3"
services:

  redis:
    image: redis:alpine
    ports:
      - "6379"
    networks:
      - frontend
    deploy:
      replicas: 2
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
  db:
    image: postgres:9.4
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - backend
    deploy:
      placement:
        constraints: [node.role == manager]
  vote:
    image: dockersamples/examplevotingapp_vote:before
    ports:
      - 5000:80
    networks:
      - frontend
    depends_on:
      - redis
    deploy:
      replicas: 2
      update_config:
        parallelism: 2
      restart_policy:
        condition: on-failure
  result:
    image: dockersamples/examplevotingapp_result:before
    ports:
      - 5001:80
    networks:
      - backend
    depends_on:
      - db
    deploy:
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure

  worker:
    image: dockersamples/examplevotingapp_worker
    networks:
      - frontend
      - backend
    deploy:
      mode: replicated
      replicas: 1
      labels: [APP=VOTING]
      restart_policy:
        condition: on-failure
        delay: 10s
        max_attempts: 3
        window: 120s
      placement:
        constraints: [node.role == manager]

networks:
  frontend:
  backend:

volumes:
  db-data:

There are 2 ways to deploy an application in compose v3 format. The first is the traditional option using docker-compose.

docker-compose up -d

The above option ignores the parameters under deploy section.

The second preferred option is to use “docker stack” approach as shown below. With this, the Docker services gets directly deployed in the Swarm mode cluster.

docker stack deploy --compose-file  vote

In “docker stack deploy”, compose file parsing is done by Docker engine itself compared to docker-compose binary handling compose file parsing. There are few options still missing in “docker stack deploy”. 1 of them is environment variables which has been addressed recently and should be available in 17.04 Docker version. Another is related to “extends” feature and is being discussed here. The general goal is get Docker engine have all features of docker-compose so that “docker stack deploy” would become the universal way to deploy compose files.

Difference between compose, stack and dab file formats:

Stack file is same as compose file with additional deploy section. Stack file is part of Compose v3 format. If we are using new Swarm mode, it is better to move to stack file format which is the same as compose v3.

Distributed application bundle(Dab) was introduced in Docker 1.12 as an experimental feature. dab file specifies container image with digest rather than defining container image with tags. dab file is auto-generated from docker-compose file. Many options of compose is not supported in dab. “docker stack” introduced in Docker 1.13 has the capabilities of compose and dab combined. My understanding is “docker stack” will make dab format obsolete. I checked with few folks in Docker team and this seems to be the case though it is not fully confirmed.

To generate dab from compose file, we can use the following command:

docker-compose bundle -o 

To deploy the bundle, we can use the docker stack command:

docker stack deploy --bundle-file  

Following is the votingapp in dab format:

{
  "Services": {
    "db": {
      "Image": "postgres@sha256:c681c72ff62b637b9ad3806a946f14450f74ee0888ca70ec1472a4e0552054c2", 
      "Networks": [
        "backend"
      ]
    }, 
    "redis": {
      "Image": "redis@sha256:95f0c9434f37db0a4f9563379184ff4895685d2381e2d81bd10dfcbcadfc095f", 
      "Networks": [
        "frontend"
      ], 
      "Ports": [
        {
          "Port": 6379, 
          "Protocol": "tcp"
        }
      ]
    }, 
    "result": {
      "Image": "dockersamples/examplevotingapp_result@sha256:83b568996e930c292a6ae5187fda84dd6568a19d97cdb933720be15c757b7463", 
      "Networks": [
        "backend"
      ], 
      "Ports": [
        {
          "Port": 80, 
          "Protocol": "tcp"
        }
      ]
    }, 
    "visualizer": {
      "Image": "dockersamples/visualizer@sha256:f924ad66c8e94b10baeaf7bdb9cd491ef4e982a1d048a56a17e02bf5945401e5", 
      "Networks": [
        "default"
      ], 
      "Ports": [
        {
          "Port": 8080, 
          "Protocol": "tcp"
        }
      ]
    }, 
    "vote": {
      "Image": "dockersamples/examplevotingapp_vote@sha256:8e64b18b2c87de902f2b72321c89b4af4e2b942d76d0b772532ff27ec4c6ebf6", 
      "Networks": [
        "frontend"
      ], 
      "Ports": [
        {
          "Port": 80, 
          "Protocol": "tcp"
        }
      ]
    }, 
    "worker": {
      "Image": "dockersamples/examplevotingapp_worker@sha256:3e4dd0f59c15f432280a2c0679c4fc5a2ee5a797023c8ef0d3baf7b1385e9fed", 
      "Networks": [
        "frontend", 
        "backend"
      ]
    }
  }, 
  "Version": "0.1"
}

How does “docker stack deploy” really work?

I have prepared the flowchart below based on referring the notes from this issue. The flowchart below shows how the image specified in the compose file resolves to what gets deployed finally when “docker stack deploy” is used.

compose2

“docker stack deploy” is idempotent. If the stack is already deployed, “docker stack deploy” will restart only those services which has the digest or tag that is updated. In the above flowchart, the restart applies to the 2 cases where the latest image is pulled from registry.

References:

Advertisements

4 thoughts on “Comparing Docker compose versions

  1. Hello, thanks for a clarifying article. As I understand it, docker stack deploy is a separate docker command, uses the docker-compose format but not docker-compose. So, I’m afraid docker-compose features like environment variables expansion, or merge of several compose files via -f flag will not work, is it right?
    Thanks in advance

    1. hi Manuel
      The goal is to get “docker stack deploy” on the same page as “docker-compose up”. There are few things missing and you have pointed out 2 important things. environment variables is fixed as part of PR(https://github.com/docker/docker/pull/30781). Might need 17.04. For “extends”, it looks like there is active discussion if “extends can be avoided. Pls refer here(https://github.com/docker/docker/issues/31101)

      Thanks
      Sreenivas

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s