Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Automated VM Deployments With the DSC VM Extension

DZone's Guide to

Automated VM Deployments With the DSC VM Extension

If you use ARM templates for your VM deploys, take a look at the DSC VM extension, which helps automatically configure your VMS.

· Cloud Zone
Free Resource

Site24x7 - Full stack It Infrastructure Monitoring from the cloud. Sign up for free trial.

Azure Resource Manager (ARM) templates are a great resource for deploying Azure infrastructure, including virtual machines, in a declarative manner. However, using an ARM template to deploy a VM will only get you as far as having a VM deployed and the operating system installed and running. The next step is to get any applications and supporting software installed on those machines. Of course, you can RDP to the machine and do this manually, but this breaks down quickly if you have lots of machines or need to regularly deploy VMs. This is where PowerShell Desired State Configuration (DSC) can come in. DSC is another declarative configuration method, but this time it is for configuring Windows and the applications that run on it. By combining ARM templates and DSC, we are able to fully automate the provisioning of a VM from creating the Azure object all the way up to installing your applications. In this post, we will look at how you can use the ARM DSC Extension to trigger your DSC configurations on a VM as part of your automated deployment.

I’m not going to go into great detail on DSC in this post. If you are interested in learning about DSC, then you can start here. It should also be noted that DSC is not the only method of deploying VM configuration — other tools like Chef, Puppet, Salt, etc. can all be used in a similar way, and many also have ARM extensions that can be deployed in a similar manner.

VM Extensions

Before we delve into the DSC extension, it is worth mentioning VM extensions in general. VM extensions are a way for application vendors (including Microsoft) to provide a pre-packaged way to deploy their software to Azure VMs. Extensions are configured to deploy to a VM, usually with some parameters set by the user and when deployed install an application in the desired configuration. Extensions are available for things like antivirus, monitoring and in this case, configuration management. Extensions can be deployed through the portal, where you can see a full catalog of available extensions, or through PowerShell, CLI, and ARM templates. Some extensions do require either bringing your own license or incurring an additional cost (things like third party AV).

Image title

Publishing DSC Files

The first thing we need to do to work with the DSC extension is to put our DSC files somewhere the extension can reach them. At the time of deployment, your ARM template will run the VM deployment and then the DSC extensions, the first thing it will do is reach out to a location you specify as a parameter and download your DSC files, so we need to put them somewhere it can access them. Unless you have VPN or ExpressRoute connectivity at deployment time this will usually need to be a public URL. The easiest way I have found to deploy my DSC resources is to an Azure storage account. That way, I can have a public URL, but I can also protect it with a SaS key if the data is sensitive.

The extension expects to find your DSC files in a ZIP file at the URL specified. One of the really good things about the extension is that it allows you to bundle your DSC file with any DSC resources it requires. That way, you don’t have to ensure any required resources are already on your VM, you just bundle them in your ZIP and the extension will take care of putting them in the right place. The structure of your zip file should look something like this:

Image title

In this ZIP file, I have two DSC resources I need in my configuration. The extension will take care of placing these in C:\Program Files\WindowsPowerShell\Modules so I can use them. DSCConfiguration.ps1 is my configuration file, which we will call to configure the VM.

Configuration File

The content of the DSC configuration file does not need to be updated especially for use with the DSC extension. Your existing configuration files should work fine. The only change you may need to make is if you wish to pass parameters from your ARM template to the DSC file, in which case your DSC file needs to be configured to accept these parameters. Again these are nothing special, just standard PowerShell parameters.

Configuring the Extension

Now that we have our ZIP file uploaded somewhere accessible, we can amend our ARM template to install the extension on the VM. The full specification for the extension ARM template is available here. The resource can sit either underneath the VM you wish to deploy it to, or as a top-level resource with a dependency on the VM it is to be deployed to. The example below is provided as a top-level resource, with a dependency.

{
    "type": "Microsoft.Compute/virtualMachines/extensions",
    "name": "[concat(parameters('vmName'),'/', variables('vmExtensionName'))]",
    "apiVersion": "2015-05-01-preview",
    "location": "[resourceGroup().location]",
    "dependsOn": [
        "[concat('Microsoft.Compute/virtualMachines/', parameters('vmName'))]"
    ],
    "properties": {
        "publisher": "Microsoft.Powershell",
        "type": "DSC",
        "typeHandlerVersion": "2.19",
        "autoUpgradeMinorVersion": true,
        "settings": {
            "configuration": {
                        "url": "[parameters('modulesUrl')]",
                        "script": "dscConfiguration.ps1",
                        "function": "Configuration"
                    },
            "Properties": {
                "SQLVersion": "[parameters('sqlVersion')]",
                "SQLAdminUserName": "[parameters('SQLAdminUserName')]"
            }
        },
        "protectedSettings": {
            "configurationArguments": {
                "AdminPassword": "[parameters('adminPassword')]"
            },
            "configurationUrlSasToken": "[parameters('sasToken')]"
        }
    }
}


There are a number of different options with this extension. The configuration above is one that should work for most, but for full details on all the options, see this page. If we take a look at the options in the properties section:

  • Published and Type: This defines which extension we want to install.
  • typeHandlerVersion: This is the version of the extension you wish to install. You can find the latest version via the portal or PowerShell. You can also use wildcards, such as “2.*”. If you use the next set to auto upgrade, you should always get the latest version regardless of what you choose here.
  • autoUpgradeMinorVersion: This determines whether the extension should automatically get upgraded upon the release of a new version. I would recommend enabling this.
  • configuration: This section contains all the details on how to download and run your DSC configuration
    • url: This is the URL where we uploaded the ZIP file with your configuration earlier. Note that if you used a SaS token with this URL, do not include it here, we will add that later.
    • script: The name of the configuration file to be run.
    • function: The name of the function inside your configuration file to be run.
  • Properties: Here we can list any arguments we want to pass through to the extension. These must exist as arguments in your DSC configuration. All proprties here are passed unencrypted to the VM, which will then be stored in your configuration file unencrypted on that VM, so do not pass any sensitive data in this section
  • Protected Settings: Here is where sensitive data can be passed. There are a couple of sections:
    • configurationArguments: This is any data you wish to be passed as arguments to your DSC configuration, but that you wish to be encrypted. The DSC file arguments should expect to receive a secure string
    • configurationURLSaSToken: If your uploaded DSC ZIP file needs a SaS token to access, this is where it can be passed.

These are the basic values that will exist in nearly all DSC extensions and should be enough to get started. There are a couple of more advanced features that could be of use to some:

ForceUpdateTag

With the configuration above, the first time you deploy your ARM template, it will deploy and run your DSC configuration as expected. If you run your deployment again, however, it will not re-run your DSC extension. It will just report the previous status. This includes if you have actually changed your DSC configuration and re-uploaded it, so you do want it to be run again. This is even the case when the installed extension resulted in an error. This is because, as far as the ARM deployment is concerned, things have not changed, none of the values in your template ARM extension are different. Yes, your DSC ZIP file has different content, but at the time of deployment, it does not know this. To get around this and ensure that you are able to re-run your DSC configuration each time, you can use the ForceUpdateTag value, as below:

"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.20",
"autoUpgradeMinorVersion": true,
"forceUpdateTag": "[parameters('DscExtensionUpdateTagVersion')]",


The value of the forceUpdateTag section can be anything you want, it just needs to differ from the previous version. In my settings, I am using Octopus Deploy to push my ARM templates, so I pass the release number here, which will differ each time.

ConfigurationData

Instead of passing any variable data through to the DSC configuration through arguments, you can instead pass a configuration data file. This is a PSD1 file that contains a hashtable of your data to be passed. You can see more details on the layout of this here. To pass configuration data, you again need to upload this to an accessible location and supply a URL. This should be a URL directly to the PSD1 file. This is configured in the settings section:

       "settings": {
            "configuration": {
                "url": "[parameters('modulesUrl')]",
                "script": "dscConfiguration.ps1",
                "function": "Configuration"
            },
            "configurationData": {
                "url": "[parameters('configurationUrl')]"
            },
            "Properties": {
                "SQLVersion": "[parameters('sqlVersion')]",
                "SQLAdminUserName": "[parameters('SQLAdminUserName')]"
            }
        },


If you wish to protect this file with a SaS token, you can again specify this in the protected settings section under the configurationDataUrlSasToken section:

     "protectedSettings": {
            "configurationArguments": {
                "AdminPassword": "[parameters('adminPassword')]"
            },
            "configurationUrlSasToken": "[parameters('sasToken')]",
            "configurationDataUrlSasToken":"[parameters('dataSasToken')]"
        }


Deployment

Now you’ve set up your extensions in your ARM template and you are ready to deploy. At this point, a deployment is no different from your standard ARM template deployment. Depending on how complex your DSC configurations are, your deployment time may increase.

Debugging

Once you test your deployment, there will invariably be issues and errors. If your DSC configuration fails, this will be reported in the Azure portal and will stop your ARM deployment from continuing. The portal will give you a brief description of the error, but often this is just something along the lines of “configuration failed to apply” rather than a useful error. To investigate the issue, you will need to log into the VM and check a few locations:

  1. C:\WindowsAzure\Logs\Plugins\Microsoft.Powershell.DSC\<DSC Extension Version Number> – This is where your verbose DSC logs will be, you can look at these and find where the error occurred
  2. C:\Packages\Plugins\Microsoft.Powershell.DSC\<DSC Extension Version Number> – This is where the extension's data is stored and there are a few places in here to look for information
    1. Status folder – In here is the latest status of the DSC extension, it’s useful to see what the date and time on this is to see if your error is actually from your recent configurationDSC or if you are just getting a cached response
    2. DSCWork – This is where your DSC file is actually extracted to and your MOF files are generated. You can look in here to see that you are actually running the right file or even manually alter and compile your DSC file if you want to test it on that machine.

Tips

  • With all extensions, including the DSC one, I have found it less error prone if you install one extension at a time. So if you are installing multiple extensions on a VM, use dependencies in your template to ensure only one runs at a time.
  • I have seen times where, even though I am using the ForceUpdateTag option, I still get a cached response rather than a re-run of my script. In this instance, you can force it to re-deploy by deleting the DSC extension from the VM and then re-running your deployment. Removing the extension does not remove any of the existing configuration changes you have deployed
  • If you're passing sensitive data as arguments, make sure you use the protected settings section to encrypt. You can check that is working by looking at your compiled MOF files in the Packages folder and checking that you can’t find the sensitive data in plain text
  • If you're following DevOps processes and checking your DSC files into source control, then using a build/release tool (VSTS, Jenkins, Octopus, etc.) to bundle up your DSC files and push them to your blob store whenever you make changes is a great way to avoid having to do this manually (and forgetting).

Site24x7 - Full stack It Infrastructure Monitoring from the cloud. Sign up for free trial.

Topics:
cloud ,arm templates ,virtual machine ,automated deployments ,tutorial

Published at DZone with permission of Sam Cogan, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}