Over a million developers have joined DZone.
Platinum Partner

OpenStack Orchestration with Heat + Devstack

· Cloud Zone

The Cloud Zone is brought to you in partnership with Mendix. Better understand the aPaaS landscape and how the right platform can accelerate your software delivery cadence and capacity with the Gartner 2015 Magic Quadrant for Enterprise Application Platform as a Service.

[This article was originally written by Shay Naeh.]

OpenStack Heat + Cloudify + Devstack

Today in order to bootstrap a Cloudify manager you have to take care of the OpenStack compute, networking and security services yourself, essentially all of the OpenStack orchestration (this is still a good reference).  Each of these is comprised of a number of configurations. 

Just to name a few:

  • When we talk about compute services, these can include provisioning the right instance type, memory and CPU characteristics, image type, and more.
  • Networking services include networks, ports, subnets, association with floating IPs as well as an establishment of a router and binding its interfaces correctly.  See this post to learn more about OpenStack Networking.
  • Security services include the creation of security groups to define which network ports are open and whom can access them, as well as the creation of security keys to SSH into the manager machine, and future instances machines. There is a different key for the manager machine and and a different key for the launched application instances.

Below is an example of the bootstrapped Cloudify manager after all the above IaaS services are provisioned.


The Heat Stack

Moreover, when creating the Cloudify manager in the usual way you have to install a CLI (command line interface), which in turn connects to Openstack endpoints like the Identity endpoint and Neutron endpoint, and communicates with them over the network using JSON-based commands. This works fine, but since there are more components involved like installing a CLI, it may take more time and is at times more error prone.

To simplify this, you can create an OpenStack Heat stack, and launch the stack which will create everything automatically. Later on I'll show a real example of how to launch the Cloudify manager as a Heat stack.

OpenStack orchestration made easy. Test drive Cloudify.     Go

First let's talk about Heat. Heat is based on a HOT template – (which we first discussed in this post on OpenStack Heat if you need a refresher),  which resembles the TOSCA standard. The template is composed of three main sections:

  • The parameters: Where you can define various parameters and constants to be referenced later when creation and accessing resources.
  • The resources section: Where you define and provision resource like compute resources, network, subnets, etc.
  • The output section: Where you can output variables like the web page URL, public IPs, etc.

Heat stacks can be nested, and one stack can reference a second stack. This is what is called Template Composition with Amazon CloudFormation.

As we shall see next, you can define IaaS services and dependencies between them, for example network creation comes before a port creation or a subnet creation.

Cloudify Manager provisioned as a Heat Stack

heat_template_version: 2013-05-23

description: >
  Cosmo manager template

    type: string
    description : Name of a KeyPair to enable SSH access to the instance
    default : shay
    type: string
    description: Instance type for management server
    default: m1.large
      - allowed_values: [m1.small, m1.medium, m1.large]
        description: instance_type must be one of m1.small, m1.medium or m1.large
    type: string
    description: ID of the image to use for the management server
    default: fc08d66a-31ba-41af-9a07-5bbf4546a7b0 
    type: string
    description: manager network address (CIDR notation)
    type: string
    description: manager network gateway address
    type: string
    description: Start of app network IP address allocation pool
    type: string
    description: End of app network IP address allocation pool
    type: string
    description: The ID of the public network. You will need to replace it with your DevStack public network ID
    default: 09c20c92-e654-480d-bce4-f0bc1f9fc785 


      type: OS::Neutron::Net
        admin_state_up: true
        name: manager_network
        shared: true
        tenant_id: admin

      type: OS::Neutron::Subnet
        network_id: { get_resource: manager_network }
        cidr: { get_param: manager_net_cidr } 
        gateway_ip: { get_param: manager_net_gateway }
        dns_nameservers: [,]
          - start: { get_param: manager_net_pool_start }
            end: { get_param: manager_net_pool_end }

      type: OS::Neutron::Router

      type: OS::Neutron::RouterGateway
        router_id: { get_resource: router }
        network_id: { get_param: public_net_id }

      type: OS::Neutron::RouterInterface
        router_id: { get_resource: router }
        subnet_id: { get_resource: manager_subnet }

    type: OS::Nova::Server
      image: { get_param: image_id }
      flavor: { get_param: instance_type }
      key_name: { get_param: key_name }
        - port: { get_resource: manager_port }
          template: |
            #!/bin/bash -v
            sudo wget http://repository.cloudifysource.org/org/cloudify3/3.0.0/nightly_1/cloudify-components_3.0.0-beta-b1_amd64.deb -P /cloudify
            sudo wget http://repository.cloudifysource.org/org/cloudify3/3.0.0/nightly_1/cloudify-core_3.0.0-beta-b1_amd64.deb -P /cloudify
            sudo wget http://repository.cloudifysource.org/org/cloudify3/3.0.0/nightly_1/cloudify-ui_3.0.0-beta-b1_amd64.deb -P /cloudify-ui
            sudo wget http://repository.cloudifysource.org/org/cloudify3/3.0.0/nightly_1/ubuntu-agent_3.0.0-beta-b1_amd64.deb -P /cloudify-agents

            sudo dpkg -i /cloudify/*.deb
            sudo /cloudify-components/cloudify-components-bootstrap.sh
            sudo /cloudify-core/cloudify-core-bootstrap.sh ubuntu manager_host  
            sudo dpkg -i /cloudify-ui/*.deb
            sudo dpkg -i /cloudify-agents/*.deb


            manager_host: {get_attr: [manager_port, fixed_ips, 0, ip_address]}

      type: OS::Neutron::Port
        network_id: { get_resource: manager_network }
          - subnet_id: { get_resource: manager_subnet }
      type: OS::Neutron::FloatingIP
        floating_network_id: { get_param: public_net_id }
        port_id: { get_resource: manager_port }
    description: URL for manager
        template: http://host
          host: { get_attr: [manager_floatingip, floating_ip_address] }

You can see above a real example of how to install the Cloudify manager as a Heat stack. In the parameters section we define several parameters, like the key to SSH to the manager machine, instance type, image ID, the manager network in CIDR notation, the gateway address, start/end network allocation pool, as well as the public network ID.

In the resources section, we first create the network, then the subnet . You can see how we reference the parameters defined in the previous section like gateway_ip. We also reference the Google name servers as the DNS name servers to resolve names to IP addresses. Next, we provision the router and bind its interfaces. And now after all of the IaaS services are provisioned, and are in place, we can provision the Cloudify manager with the right image_id, flavor, and key_name. In the user data we create a bash script which installs the manager packages. First it downloads the right packages from the net, and then installs them.

In the output section we retrieve and present the manager’s floating IP address.

Below are two charts, the first shows how the heat stack looks graphically, with all of the connected nodes and dependencies.

The second chart shows the provisioned resources:  Networks, subnets, Floating IPs and Cloudify manager VM.



In order to access the Cloudify manager from an external browser I've created port forwarding on the Devstack machine, so that when you access port 8080 of the Devstack VM hosted on HP Cloud, you will be forwarded to port 80 on the Cloudify manager provisioned VM (this is specific to to this current example, but this can easily be provisioned on any Openstack cloud). The commands to create the port forwarding are listed below.


In the chart below you can see a screenshot of how the bootstrapped Cloudify manager looks when it’s up and running.


The Cloud Zone is brought to you in partnership with Mendix. Discover how Mendix for Mobile App Deveopment makes it incredibly fast and easy for any developer to build engaging multi-channel apps.


Published at DZone with permission of Cloudify Community , DZone MVB .

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}