Category Archives: openstack

Hybrid cloud recent solutions from Microsoft and VMWare – 2 different ends of the hybrid cloud spectrum

Public clouds have grown tremendously over the last few years and there are very few companies who do not use public cloud at this point. Even traditional enterprises with in-house data centers have some presence in the public cloud. I was looking at Amazon’s re:Invent conference details and I was amazed by the number of new services and enhancements that were announced this year.  It is very difficult for private clouds to keep up in pace with the new features of public cloud. There is no doubt that public clouds will overtake private clouds in the long term. Private clouds still have a wide deployment and there will be enough use cases for quite some time to deploy private cloud. The use cases includes regulated industries, compute needed in remote locations not having access to public cloud and some specialized requirements that public clouds cannot meet. For some enterprises, private cloud would make more sense from a costing perspective. Having hybrid cloud option is a safe bet for most companies as it provides the best of both worlds. I saw 2 recent announcements in hybrid cloud that captured my attention. One is Azure stack that allows running Azure stack in private cloud. Another is VMWare cloud on AWS that allows running entire VMware stack in AWS public cloud. I see these two services as 2 ends of the hybrid cloud spectrum. In 1 case, public cloud infrastructure software is made to run on private cloud(Azure stack) and in another case, private cloud infrastructure software is made to run on public cloud(Vmware cloud on AWS). In this blog, I have tried to capture more details on these 2 services.

Hybrid cloud

There are predominantly 2 options currently to run Private cloud. 1 option is to use vendor based cloud management software along with hardware from same vendor. Cisco UCS is an example in this category where customers get to use UCS servers integrated with networking and storage along with management software from Cisco. This provides a tightly integrated solution. Another option is to use Openstack as a cloud orchestration system and use server, networking and storage hardware from any vendor. Both these kinds of solutions works well in private cloud. For enterprises having private cloud, there always are use-cases where few services makes more sense in public cloud. A classic example is a development team using public cloud for 1 of their development projects for agility reasons. Once the development is complete, operations team has a choice to deploy the application in either private or public cloud. There is also the use case where current applications deployed in private cloud needs to be scaled by using public cloud for elasticity reasons. In either of the cases, we need a solution that allows easier migration of applications along with their policies between the private and public cloud.

Following are some important requirements of hybrid cloud:

  • Having a common management software to manage public and private clouds.
  • Ability to move applications seamlessly between the clouds.
  • Secure connectivity between the clouds.

Microsoft Azure stack

Azure stack is a hybrid cloud platform product from Microsoft that allows managing the private cloud with the same Azure public cloud software stack.

Following picture from Azure stack link shows the components of Azure stack:


Following are some details on the solution:

  • Azure stack takes some of the components of Microsoft Azure public cloud to manage private cloud. To start with, Azure stack will support limited services in private cloud when compared to Azure public cloud.
  • Cloud infrastructure layer is hardware and basic system software for running compute, storage and networking. In the initial release, Azure stack will be provided as turnkey integrated solution with hardware integrated from Dell, HP and Lenovo. It looks like more vendors will be added in future. The reason to support limited vendors is to achieve tight integration and simplify the deployment solution.
  • Azure infrastruture layer sits on top of cloud infrastructure and the Azure services layer interacts with Azure infrastructure layer.
  • The first technical preview release was done in early 2016 and the second technical preview release was done in late 2016. GA release is planned middle of 2017.
  • The entire Azure stack runs currently on a single node. The plan is to make this distributed in future.

Following are some of my general thoughts on Azure stack:

  • Public cloud providers typically did not focus on private clouds since that would eat into their pie. This is a good move by Microsoft to facilitate hybrid cloud and the gradual move to public cloud.
  • The pricing and licensing model of Azure stack is not clear. Since the plan is to have a turnkey integrated solution with few vendors, there has to be some form of licensing agreement with multiple parties.
  • It is not clear how the OEM vendors providing cloud infrastructure can differentiate their solutions.
  • Having a restricted cloud infrastructure vendor list will make this solution not useful for private clouds using legacy hardware. It will be good if the cloud stack can provide common API that can allow any hardware vendor that supports the API to be managed by the Azure stack cloud software. To some extent, Openstack is following this model. It will be good if Azure stack can do the same so that there is no restriction on the vendor list.
  • AWS and Google cloud have not introduced private cloud management solutions till now. As mentioned earlier, there are use cases where access to public cloud is not possible and private cloud would be a better fit. AWS greengrass IoT solution is the closest private cloud like solution from AWS that I have seen where local IoT resources are used for compute when needed.

VMWare cloud on AWS

This solution allows the entire VMWare virtualization stack(including compute, storage and networking) to run in AWS public cloud. The solution is provided by VMWare and is a joint collaboration between VMWare and Amazon AWS. For enterprises using VMWare stack to manage their private cloud infrastructure, they can use the same software stack when moving some of their services to AWS public cloud.

Following picture from VMWare link shows the components of this solution:


Following are some details on the solution:

  • All the core components of VMWare stack including Vsphere, Virtual SAN, NSX, ESX and Vcenter runs in AWS infrastructure.
  • AWS typically uses Xen hypervisor for virtualization and VMWare uses ESX for virtualization. To do this integrated solution, ESX runs in AWS baremetal. There is no Xen hypervisor in this integrated solution.
  • Vcenter is used for management across on-premise as well as in AWS. In 1 of the joint demos, VMWare shows seamless VM migration between on-premise cloud and AWS cloud.
  • VMs deployed in AWS public cloud can use all the AWS services like storage, database, analytics etc. This makes this solution very attractive.
  • This service will be operated and managed by VMWare. Both AWS and VMWare have made changes to their stack for this integrated solution.
  • The solution is currently in Technical preview phase and general availability is expected middle of 2017.

Following are some of my general thoughts on this VMware cloud on AWS solution:

  • VMWare has tried different strategies to get a foothold into public and hybrid cloud. vCloud hybrid service was 1 of their unsuccessful attempts earlier on this. This solution will benefit both VMWare and AWS, the bigger benefit lies for AWS.
  • AWS has not sold baremetal servers till now. There are companies like Packet that provides baremetal servers. There are use cases for baremetal like non-virtualized scenarios or pure container based solutions where baremetal servers would help. It will be interesting to see if AWS would sell these baremetal servers in future. It is not clear why AWS has not provided bare metal servers till now, 1 possible reason could be that it would take away some of its differentiators.
  • Microsoft has a private cloud enterprise solution with hyperv and public cloud solution with Azure public stack. Microsoft can provide a similar integrated solution that allows Microsoft’s private cloud stack to run on its Azure public cloud. It is not sure if Microsoft will venture into this.


Both solutions described above are good hybrid cloud solutions that eases movement to public cloud. Both these hybrid cloud solutions are favorable more for public cloud rather than private cloud. Even though these solutions helps private clouds temporarily, long term benefits lies with public cloud. It will be good to have cloud management software that is cloud agnostic so that multiple cloud vendors can be used and there is no vendor lock-in. Terraform and Cliqr are some solutions catered to this space.


Openstack Deployment using Containers

I recently saw the Openstack self-healing demo from CoreOS team using Tectonic(Stackanetes project) and I kind of felt that the boundary between Containers and VMs are blurring. In this blog, I discuss the usecase of deploying Openstack using Containers.

We typically think of Openstack as a VM Orchestration tool. Openstack is composed of numerous services and deploying Openstack as one monolithic blob is pretty complex and difficult to maintain. The demo described showed how Containers simplify Openstack deployment. This is a great example of using Microservices architecture to simplify infrastructure deployment.

Following diagram shows the Openstack deployment model using Containers. The diagram below shows how Openstack service containers deploys user VM. The user VMs deployed using Openstack can run Containers as well..


Following are some notes on the architecture:

  • Openstack services like Nova, Heat, Horizon are containerized using Openstack Kolla project as Docker Containers. Some Openstack services like Nova is composed of multiple Containers.
  • Infrastructure components like Ceph, Openvswitch, Mongodb are also deployed as Containers.
  • For Container deployment, Openstack natively uses Ansible. Kubernetes can also be used for Orchestration.
  • Using Containers for Openstack service containers gives all the build, ship and deploy advantages of Containers.
  • Using orchestration solution like Kubernetes gives all the resiliency and deployment advantages for Openstack services.

This work also shows how Containers and VMs can work closely with each other for lot of use-cases. There are other Openstack projects like Magnum and Kuryr where there is an intersection between Containers and VMs. Magnum project deals with Container orchestration using Openstack and Kuryr project deals with doing Container networking using Openstack Neutron.


Vagrant and Devstack

Openstack is a Cloud Orchestration software. Devstack script provides a development environment for Openstack. Devstack provides a great way to get hands-on with Openstack. I had written 2 earlier blogs on installing Devstack for Openstack Icehouse and Openstack Juno. I received multiple queries on installation related issues. To make this simple, I created Vagrant images for different Openstack releases. With this, VM creation and Devstack installation can all be done with a single script. In this blog, I will walk-thru the steps for the installation.

Vagrant makes it easier to create and share VMs and this makes Vagrant Devops friendly. For getting started on Vagrant, you can refer  to my earlier blog on Vagrant.

My Development environment:

Windows 7 machine with Virtualbox 4.3.28 and Vagrant 1.7.2.


Following are typical issues I have seen folks facing when running Devstack:

  • There are some pre-requisite software that needs to be installed before running Devstack like setting up Python environment etc.
  • It is needed to setup VM with atleast 4G RAM and 8G hard disk. Otherwise, either Stacking will fail or instance creation will fail.

Continue reading Vagrant and Devstack

Openstack and Docker – Part 2

This is a continuation of my previous blog on Openstack and Docker. In this blog, I will cover Openstack Docker heat plugin and Magnum.

Following are some of the items that Nova Docker driver cannot do currently:

  1. Passing environment variables
  2. Linking containers
  3. Specifying volumes
  4. Orchestrating and scheduling the containers

Heat docker plugin solves problems 1-3 and partially solves problem 4. Following is the architecture diagram I found in Openstack Docker wiki for heat.


  • Nova is not involved here. Openstack heat uses Docker plugin to talk to Docker agent on the host.
  • The host here is the VM spawned. The VM can either be spawned by Nova or Heat can spawn this using Nova driver.
  • Glance is not involved here as the container images are stored in Docker registry.
  • The Heat approach allows us to specify environment variables, link containers, specify volumes as well as orchestrate the host on which the Docker runs.

Using Heat plugin:

Continue reading Openstack and Docker – Part 2

Openstack and Docker – Part 1

In this blog, I will cover the different ways in which Openstack can create and manage Docker Containers. The 3 predominant approaches are using Nova Docker driver, Heat Docker plugin and Magnum. Magnum is pretty new and is under development. Openstack is opensource cloud orchestration software and Docker is opensource container management software. For this blog, I am assuming users are already familiar with Openstack and Docker. There are lot of resources for learning Openstack and Docker available in the web, my blogs related to these topics can be found here and here.

Nova Docker Driver:

Nova typically manages VMs. In this approach, Nova driver is extended to spawn Docker Containers. Following is the architecture diagram mentioned in the Nova Docker wiki.


  • To spawn containers Nova compute driver is pointed to Docker driver.
  • Nova Docker Virt driver talks to Docker agent using http api calls.
  • Docker images are stored in the Docker registry and images are exported to glance from Docker registry which Nova uses to create Containers.

Nova Docker driver with Devstack:

Continue reading Openstack and Docker – Part 1

VIRL and CML – Overview

CML(Cisco Modeling lab) and VIRL(Virtual Internet and Routing lab) are Network modeling platforms from Cisco. I have been trying this out for the last 2 weeks and I am very impressed by what it can do. I feel that the potential for this platform is so huge that it will create a fundamental impact in the Networking industry. Currently, the simulation is limited mainly to Cisco devices though I have seen Juniper and Vyatta images in the VM list. In the next series of blogs, I will walk through some of the following topics:

  • What is CML, VIRL? Architecture and Software components.
  • How to get started? Installation and Quickstart.
  • Examples/Use cases that I tried. L3, L2, management.

Difference between CML and VIRL:

CML is a code branch of VIRL that has been enhanced to provide more scale, VIRL has a 15 node limit. CML customers get Cisco TAC support. CML is focussed towards Enterprise customers, while VIRL is focussed on individuals and training institutions. Obviously, VIRL is much cheaper than CML. VIRL has different prices based on personal or academic use.

Since I work in Cisco, I didnt have to pay any money to try out CML and VIRL. Going forward, I will use the term VIRL to describe the Network Modeling platform. If there are any CML specifics, I will mention those.

VIRL/CML Overview:

Continue reading VIRL and CML – Overview

Openstack Juno – Management interfaces

This blog is part of my series on Openstack Juno. In this blog, I will cover different management interfaces to Openstack. Following are the different management interfaces available.

  • Horizon web interface
  • CLI interface to each service. CLI interface is provided by Python script. Internally, the script calls the REST interface.
  • REST interface. This is accessible either through Curl or a POSTMAN kind of client.
  • Programmatic interface using Python SDK.

Horizon interface:

On the host where stacking is done, webserver runs on port 80 and all Openstack services can be configured using this interface. Login to Horizon can be done with either tenant userid or admin userid. Based on the userid, privileges are granted.

CLI interface:

CLI interface is provided for each service. Nova services are accessible through “nova” client, Swift services are accessible with “swift” client and so on. Following example lists running VMs. Continue reading Openstack Juno – Management interfaces