Azure Resource Manager Templates and Nested Loops: A Commentary

What is Azure Resource Manager, or ARM?

ARM is a Microsoft Azure provided managed service that enables an automation designer to define their design intent, expressed as templates, using ARM's automation language, which is currently AzureRM and is transitioning to Az.

ARM enables a designer to express their intent as to the selection, configuration, and assembly sequence of computing resource objects as specified by ARM automation resource templates.

For example, an automation designer's intent to design the resource objects and their necessary sequencing in order to install 0toN virtual machine extension resource objects on one to n existing virtual machine resource objects.

When Is This Automation Used?

This automation is used when an automation designer needs to provision multiple provider-supplied virtual machine extensions on multiple existing, running Azure virtual machines. For example, to provision Microsoft-supplied anti-malware and/or BgInfo virtual machine extensions on existing virtual machines.

An automation designer uses Azure Resource Manager’s (ARM's) declarative template language to implement an automation’s design as the specification of each automation resource, and the order in which automation resources are deployed or provisioned.

When initiated, each resource specification is rendered, by ARM, as a resource object, in an Azure subscription namespace. The Azure subscription namespace is secured by Azure active directory and observable through the Azure portal.

Design: Activity Composition

An automation designer expresses an automation’s design as the composition and sequencing of activities that add and/or change the arrangement of resource objects in the automation’s namespace, in an Azure subscription. Activities can be composed and are implemented using Azure tasks. ARM tasks are not composable, but ARM templates are.

The activities in this automation install 0 to n virtual machine extensions, on 1 to n virtual machines. The automation is parameterized by two sets, a virtual machine names set, and a virtual machine extension names set. Both parameters are structured as arrays of strings.

Design: The Automation’s Anatomy

This automation has three levels of activities. The first two levels have iteration controls and are composed with the unique, provider-supplied virtual machine extension resource specifications at the third level.

The activity compositions are:

• Level 1, the user-initiated activity, iterates over the virtual machine names in the array vmNames. A resource condition statement stops composition with level 2 when array providerVmExtensionNames  is empty.
  • See template install-0toN-pVmExts-on-1toN-vms.json
• Level 2 activity is initiated from level 1 and iterates over the designer nominated virtual machine extension names in the providerVmExtensionNames  array. See template install-0toN-pVmExts.json
• Level 3, is initiated from level 2 and installs each virtual machine extension specification resource on a nominated virtual machine.
  • See provision-pVmExt-antiMalware.json or provision-pVmExt-bgInfo.json

Each level 3 activity is a provider-supplied, virtual machine extension resource specification used to provision an anti-malware or bgInfo vm extension on to the already-installed virtual machine resource objects tpt-ar-20. tpt-ar-21.

Design: Activity Composition and Nested Iteration Schematic

The image above shows activity composition over two levels implemented by nested template links, activity iteration, and the empty array condition at level 1 that stops composition with level 2.

Implementation: Composition Control, the Challenge

ARM’S declarative, template language enables an automation designer to implement template structures comprised of individual ARM templates, corresponding to each design activity, to control the sequence (i.e. order), the number and, the location — in a subscription’s namespace — of the resource objects comprising an automation.

For example, the designed activity install-0toN-pVmExts-on-1toN-vms included in this automation, uses a three level nested template structure with four templates, two at level 3. The template structure corresponds to the designer’s arrangement of the automation’s activities.

The happy path through the three level nested and iterated template structure emerges when the virtual machine names array and the virtual machine extension names array both have members.

The level 1 template fails when the virtual machine names array, vmNames , is empty. The ARM engine throws a template validation error:

This image shows an invalid template error message caused by an empty array evaluated by a copy element’s count property.

Inserting a resource condition element does not prevent the error.

 "resources": [
    {
      "condition": "[not(variables('pVmExtNamesIsEmpty'))]",
      "type": "Microsoft.Resources/deployments"
    }
  ]

The code block above, install-0toN-pVmExts-on-1toN-vms.json, shows a resource condition element in template for level 1 composition activity

Implementation: Composition Control, the Diagnosis

It appears that in templates nested with a resource template link element, the resource condition element (above), is applied by ARM’s template validation engine, after the engine has evaluated a resource copy element’s count property.

As a consequence, the copy element’s count property is evaluated before the resource condition element is applied, and ARM returns the above error. For example, as in template install-0toN-pVmExts.

  "resources": [
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2016-02-01",
      "name":  "[concat('Install-providerExtensions-ForVm-', parameters('vmName'), '-', variables('pVmExtNames')[copyIndex()])]",
      "copy": {
        "name": "installPVmExtLoop",
        "count": "[variables('pVmExtNamesCount')]",
        "mode": "serial"
      },
      "properties": {
        "mode": "Incremental",
        "templateLink": {
          "uri": "[concat(parameters('assetLocation'), concat('/toolsAzure/nested-iterations/nested/provision-pVmExt-', variables('pVmExtNames')[copyIndex()], '.json'), parameters('containerSasToken'))]",
          "contentVersion": "1.0.0.0"
        }
      }
    }
  ],

The resource copy element, count property in template install-0toN-pVmExts is evaluated before the condition element in template install-0toN-pVmExts-on-1toN-vms.json is applied when array providerVmExtensionNames  is empty.

Note a term in the URI concat()   function is the container sas token.

That situation is avoided in the install-0toN-pVmExts-on-1toN-vms template because that template requires the array vmNames to have at least one member, and so can not be empty. Otherwise, what’s the point.

However, the virtual machine extension names array, i.e. providerVmExtensionNames , can be empty, in which case ARM throws a similar error in the install-0toN-pVmExts template.

In summary, using a resource condition element in a deployment template to avoid the evaluation of that resource's copy element count property didn’t work for me.

Implementation: Composition Control, the Workaround

The preferred objective is to pass activity install-0toN-pVmExts-on-1toN-vms either an empty providerVmExtensionNames array or an array with members and have the activity composition complete successfully.

This automation design works around the above situation — the designer’s need to stop resource iteration and activity composition when an empty argument array is detected — by spoofing ARM‘s template validation.

The install-0toN-pVmExts template implements two variables, pVmExtNames  and pVmExtNamesCount , each variable is conditioned by a template expression  if()  function that sets the template variable’s value differently depending on the presence or absence of virtual machine extension array members.

  "variables": {
    "saIdentifiers": "[parameters('saIdentifiersVar')]",
    "pVmExtNamesRaw": "[variables('saIdentifiers').providerVmExtensionNames.value]",
    "pVmExtNamesIsEmpty": "[if(empty(variables('pVmExtNamesRaw')), json('true'), json('false'))]",
    "pVmExtNames": "[if(variables('pVmExtNamesIsEmpty'), concat(variables('pVmExtNamesRaw'), array('dummyVmExt')), variables('pVmExtNamesRaw'))]",
    "pVmExtNamesCount": "[if(variables('pVmExtNamesIsEmpty'), 1, length(variables('pVmExtNames')))]"
  }

The variables section of template install-0toN-pVmExts.json shows variables used to spoof template validation.

Note: template variable pVmExtNamesCount  is assigned int 1 when the array is empty.

•  pVmExtNamesIsEmpty: [if(empty(variables('pVmExtNamesRaw')), json('true'), json('false'))]

•  pVmExtNames: [if(variables('pVmExtNamesIsEmpty'), concat(variables('pVmExtNamesRaw'), array('dummyVmExt')), variables('pVmExtNamesRaw'))],

•   pVmExtNamesCount: [if(variables('pVmExtNamesIsEmpty'), 1, length(variables('pVmExtNames 

Note, the  concat()  function in the template variable vmExtNames   assignment returns an array, and is passed two arguments, each structured as a strings array.

The template variable values assigned in template install-0toN-pVmExts, when array pVmExtNames is empty, are irrelevant as the template install-0toN-pVmExts, is never initiated.

The resource condition element, in the template install-0toN-pVmExts-on-1toN-vms template, stops the activity composition sequence at level 1, and the level 2 template install-0toN-pVmExts is never initiated when the pVmExtNames array is empty.

  "resources": [
    {
      "condition": "[not(variables('pVmExtNamesIsEmpty'))]",
      "type": "Microsoft.Resources/deployments"
    }
  ]

Thishows the level 1 template’s resource condition element showing the condition statement that stops composition before level 2 is initiated.

Verify: Automation Input Parameters

In addition to the usual asset location and shared access signature, required for nested templates by ARM, the arrays controlling composition are shown here:

  "parameters": {
    "saIdentifiersVar": {
      "type": "object",
      "defaultValue": {
        "vmResourceGroupName": {
          "type": "string",
          "value": "tpt-algoR-vms-rg"
        },
        "vmNames": {
          "type": "array",
          "value": [
            { "name": "tpt-ar-20" },
            { "name": "tpt-ar-21" }
          ]
        },
        "providerVmExtensionNames": {
          "type": "array",
          "value": ["antiMalware", "bgInfo"]
        }
      }
    }

This shows template parameter arguments for template install-0toN-vmExts-on-1toN-vms.

Verify: Automation Template Configuration

Note: Templates for the four activities — install-0toN-pVmExts-on-1toN-vms, install-0toN-pVmExts, provision-pVmExt-bgInfo, provision-pVmExt-antiMalware — described above are available here.

For provision testing without existing virtual machines, set the value of template variable spike to true  in each provider supplied virtual machine extension template e.g. provision-pVmExt-antiMalware. Doing so prevents the virtual machine extension resource being provisioned and so stops the ARM engine looking for the previously installed virtual machine resource objects set in parameter vmNames, of template install-0toN-pVmExts-on-1toN-vms.

 "variables": {
    "spike": "[bool('false')]",
    "vmName" : "[parameters('virtualMachineName')]"
  },
  "resources": [
    {
      "condition": "[not(variables('spike'))]",
      "type": "Microsoft.Compute/virtualMachines/extensions"
    }
  ]

This shows template variable spike, set to false, and corresponding resource condition.

To demonstrate the interaction between a resource condition and a resource copy element at different activity composition levels, set the argument of parameter providerVmExtensionNames to empty.

        "providerVmExtensionNames": {
          "type": "array",
          "value": []
        }

This shows the argument of parameter providerVmExtensionNames set to empty.

The release pipeline action install-pVmExts-on-vms can be initiated from either powershell or an Azure DevOps release pipeline's blade via the Azure DevOps portal  -- was vsts portal .

This shows the Azure DevOps release pipeline action, install-pVmExts-on-vms, comprised of five tasks, three disabled. Tasks AzureBlob File Copy and install-0toN-pVmExts-on-1toN-vms provision multiple pVmExts resources on 1toN not necessarily existing virtual machines.

The AzCopy activity requires an existing Azure storage account, and a shared access signature secured blob collection container.

On successful completion of the ARM release pipeline action, install-pVmExts-on-vms.

This shows the successful completion of the azdo release pipeline action, install-pVmExts-on-vms.

Verify: Automation State Changes Observable Through Azure Portal

For provisioning, the virtual machine resource objects identified by the vmNames argument’s values must be created prior to the initiation of activity install-0toN-pVmExts-on-1toN-vms.

This shows the nominated, existing, virtual machine resource objects with successfully provisioned, Microsoft-supplied, virtual machine extension resource objects.