Over a million developers have joined DZone.

End-to-End IaaS Workload Provisioning in the Cloud

· Cloud Zone

Download the Essential Cloud Buyer’s Guide to learn important factors to consider before selecting a provider as well as buying criteria to help you make the best decision for your infrastructure needs, brought to you in partnership with Internap.

In prior articles, we've discussed Azure Automation, Azure VM Agent extensions and PowerShell Desired State Configuration (DSC) as individual tools and approaches for automated workload provisioning.  I'm frequently asked about how these tools can be used together to orchestrate end-to-end provisioning of cloud workloads, including fabric, OS and application components.

image

This article is part 1 of a two-part series. 

In this article, I'll provide an example of leveraging Azure Automation runbooks and PowerShell Workflows in a concerted approach to fully automate all aspects of provisioning new IaaS workloads on the Microsoft Azure cloud platform.

In part 2 of this series, I'll provide an example of a working PowerShell DSC Configuration that can be used with Azure Automation runbooks to customize the configuration of operating system and application components running inside each  VM as part of the provisioning process.

What, when and where?

We certainly don't have a lack of automation technologies these days! In fact, one could argue that we have so many different tools for automation, that sometimes it can be confusing to know which tools to leverage for a particular scenario. As I've been helping customers with automated provisioning sequences, here's an approach that I've found works well:

  1. Orchestrate overall provisioning process with Azure Automation
  2. Provision cloud fabric resources with PowerShell Workflow activities using Azure Automation Runbooks
  3. Configure OS and Application payloads inside Azure VMs with PowerShell DSC
  4. Publish PowerShell DSC Configuration to Azure Storage Account
  5. Use Azure VM Agent extensions to apply PowerShell DSC Configuration

In the near future, Azure Resource Manager (ARM) will also be worthwhile including in your automation toolkit to manage cloud resources as a related group of dependencies. However, currently automation of ARM is limited to a subset of Azure PaaS resources.  Azure IaaS support is coming soon, so be sure to watch this space for updates!

In the next few sections of this article, I'll step through the details around steps #1 and #2 above.  In part 2 of this two-part series, I'll finish-up with coverage of steps #3, #4 and #5.

Prepare your Azure subscription

The example in this article assumes that you already have a few basic IaaS components provisioned in your Azure subscription with a consistent naming convention, including:

  • Affinity group: xxxlabag01
  • Storage account: xxxlabstor01
  • Virtual network: xxxlabnet01
  • Windows Server Active Directory VM: xxxlabad01

When provisioning these components, replace xxx with your unique initials to arrive at a consistent naming convention that provides globally unique names for the storage account and cloud service for the Windows Server Active Directory domain controller VM. If you need help provisioning these IaaS components, check out the following resources to help you get started:

Orchestrate overall provisioning process with Azure Automation

Azure Automation provides a reliable, scalable platform service for orchestrating long-running processes that potentially touch several services and resources. Automation sequences are built as runbooks that include PowerShell Workflow activities. By leveraging PowerShell Workflows, Azure Automation runbooks can be suspended, resumed and recover from unexpected occurences while preserving current state of completed activities within the runbook.

If you haven't yet setup Azure Automation in your Microsoft Azure subscription or need some help with the basics of creating runbooks, check out this step-by-step article to walk through the process for getting started ...

Once your Azure Automation account is setup and ready to go, let's create a new Azure Automation runbook for provisioning our new cloud application, which we'll call MyWebApp in this article.

    workflow New-MyWebAppDeployment
    {
    }

As mentioned above, Azure Automation runbooks leverage PowerShell Workflows, which is why we're defining our new runbook with the workflow keyword above.

  • Tip! For step-by-step guidance on the basics of creating new runbooks, be sure to reference the Getting Started article linked above for a quick primer.

In an effort to make this runbook reusable for a variety of provisioning scenarios, I'll define a set of parameter inputs at the top of the workflow. Azure Automation will prompt for user input for each defined parameter when manually starting this runbook, or you can supply these parameter values when programmatically invoking this runbook from another runbook or PowerShell script.

    workflow New-MyWebAppDeployment
    {       
        param(
           
            [parameter(Mandatory=$true)]
            [String]
            $subscriptionName,
           
            [parameter(Mandatory=$true)]
            [String]
            $deploymentPrefix,
           
            [parameter(Mandatory=$true)]
            [Int]
            $numberOfVMs,
           
            [parameter(Mandatory=$true)]
            [String]
            $vmInstanceSize,
           
            [parameter(Mandatory=$true)]
            [String]
            $vmDomain,
       
            [parameter(Mandatory=$true)]
            [String]
            $vmDomainNetBIOS     
        )
    }

Let's also declare a set of variables near the top of our workflow for common values that will be used when provisioning virtual machines,

    workflow New-MyWebAppDeployment
    {

           ... continued from above ...

           # Get Azure credential for authenticating to Azure subscriptions
       $cred = Get-AutomationPSCredential -Name 'AzureAutomationAccount'

       # Get vmAdmin credential for admin access inside each VM
       $vmAdmin = Get-AutomationPSCredential -Name 'AzureAdmin'
      
       # Set common variable values for provisioning each VM
       $storageName = $deploymentPrefix + 'stor01'
       $vmServiceName = $deploymentPrefix +'app'
       $affinityGroupName = $deploymentPrefix + 'ag01'
       $availabilitySetName = $deploymentPrefix + 'app'
       $vNetName = $deploymentPrefix + 'net01'
       $subnetName ='Subnet-1'
       $dscArchive = 'AADSCWebConfig.ps1.zip'
       $dscConfigName = 'WebSiteConfig'

    }

Now that we've got the initial portion of our runbook created, let's save it as a draft and continue with adding the code that will provision our VM and application.

Provision cloud fabric resources with PowerShell Workflow activities

In the param section of our runbook workflow above, you'll note that we defined a $numberOfVMs parameter to accept, as input, the number of load-balanced VMs that we'd like to provision for our application.  To provide flexibility in provisioning a variable number of VMs, we'll add a for loop in our workflow that will handle the provisioning for each VM.

    workflow New-MyWebAppDeployment
    {

          ... continued from above ...

       for ($i=1; $i -le $numberOfVMs; $i++)
       {

       }

    }

As each iteration of the for loop above begins, we'll add code to checkpoint current workflow state and connect to the desired Azure subscription.

    workflow New-MyWebAppDeployment
    {

           ... continued from above ...

       for ($i=1; $i -le $numberOfVMs; $i++)
       {

            # Save current state of workflow
            Checkpoint-Workflow
          
            # Connect to Azure subscription
            Add-AzureAccount -Credential $cred

            Select-AzureSubscription `
              -SubscriptionName $subscriptionName
          
            # Set current Azure Storage Account
            Set-AzureSubscription `
              -SubscriptionName $subscriptionName `
              -CurrentStorageAccountName $storageName

       }

    }

Cloud workload provisioning tasks could involve many different cloud resources and be relatively long-running sequences.  Taking the above approach of checkpointing workflow state and reconnecting to the desired Azure subscription with each iteration of our for loop will provide the ability for this runbook to gracefully resume in the event that the workflow was suspended or unexpectedly interrupted.

Next, let's add our code for provisioning each VM inside the for loop we've defined above. Since provisioning a new Azure VM can involve several individual activities to build the desired VM configuration, we'll group all of these related steps into a single InlineScript workflow activity.

    workflow New-MyWebAppDeployment
    {

           ... continued from above ...

       for ($i=1; $i -le $numberOfVMs; $i++)
       {

            Checkpoint-Workflow
          
            # Connect to Azure subscription
            Add-AzureAccount -Credential $cred

            Select-AzureSubscription `
              -SubscriptionName $subscriptionName
          
            # Set current Azure Storage Account
            Set-AzureSubscription `
              -SubscriptionName $subscriptionName `
              -CurrentStorageAccountName $storageName

            # Provision VM
            InlineScript {
        
                 $currentVM = "{0:D2}" -f $Using:i
                 $vmName = $Using:deploymentPrefix + 'app' + $currentVM
                 $vmImage = @((Get-AzureVMImage |
                   Where-Object Label -like `
                   "Windows Server 2012 R2 Datacenter*").ImageName)[-1]
                 $vmAdmin = $Using:vmAdmin
         
                 # Specify VM name, image and size
                 $vm = New-AzureVMConfig `
                   -Name $vmName `
                   -ImageName $vmImage `
                   -InstanceSize $Using:vmInstanceSize
        
                 # Specify VM local admin and domain join creds
                 $vm = Add-AzureProvisioningConfig `
                   -VM $vm `
                   -WindowsDomain `
                   -AdminUserName $vmAdmin.Username `
                   -Password $vmAdmin.GetNetworkCredential().Password `
                   -JoinDomain $Using:vmDomain `
                   -Domain $Using:vmDomainNetBIOS `
                   -DomainUserName $vmAdmin.Username `
                   -DomainPassword `
                     $vmAdmin.GetNetworkCredential().Password
            
                 # Specify load-balanced firewall endpoint for HTTPS
                 $vm = Add-AzureEndpoint `
                   -VM $vm `
                   -Name 'WebHTTPS' `
                   -LBSetName 'LBWebHTTPS' `
                   -DefaultProbe `
                   -Protocol tcp `
                   -LocalPort 443 `
                   -PublicPort 443
        
                 # Specify VNet Subnet for VM
                 $vm = Set-AzureSubnet `
                   -VM $vm `
                   -SubnetNames $Using:subnetName
            
                 # Specify HA Availability Set for VM
                 $vm = Set-AzureAvailabilitySet `
                   -VM $vm `
                   -AvailabilitySetName $Using:availabilitySetName
     
                 # Provision new VM with specified configuration
                 New-AzureVM `
                   -VMs $vm `
                   -ServiceName $Using:vmServiceName `
                   -VnetName $Using:vNetName `
                   -AffinityGroup $Using:affinityGroupName `
                   -WaitForBoot
     
             }

        }

    }

At this point, you can save and test your new runbook.  When testing the runbook, you'll be prompted to input values for each parameter we defined. When prompted for a DeploymentPrefix, enter xxxlab, where xxx is your unique initials used at the top of this article. This prefix will be prepended to the Azure resource names used within the runbook to reference your unique naming convention.

After the runbook completes, you will have multiple VMs in the specified Azure subscription, storage account, affinity group and virtual network.  Assuming you have a Windows Server Active Directory domain controller VM running on that same virtual network, each new VM will also be joined as a member server to the specified Active Directory domain.

What's next?

In part 2 of this series, we'll walk through wiring in PowerShell DSC to configure the operating system and applications inside the VM as part of the provisioning process.  Stay tuned!

The Cloud Zone is brought to you in partnership with Internap. Read Bare-Metal Cloud 101 to learn about bare-metal cloud and how it has emerged as a way to complement virtualized services.

Topics:

Published at DZone with permission of Keith Mayer, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}