Azure Stack TP Overview, Preview, Review (Part 2)

DZone 's Guide to

Azure Stack TP Overview, Preview, Review (Part 2)

Now that you've got the basics, it's time to install Azure Stack TP3. Learn all about the configurations you can use to deploy on physical or virtual machines.

· Cloud Zone ·
Free Resource

server storage I/O trends

This is part two of a two-part series looking at Microsoft Azure Stack with a focus on my experiences installing Azure Stack Technical Preview 3 (TP3) including into a nested VMware vSphere ESXi environment. Read part one here, which provides a general overview of Azure Stack.

Azure Stack Review and Install

Being familiar with Microsoft Azure public cloud, having used it for a few years now, I wanted to get some more insight, experience, expand my tradecraft on Azure Stack by installing TP3. This is similar to what I have done in the past with OpenStack, Hadoop, Ceph, VMware, Hyper-V, and many others, some of which I need to get around to writing about sometime. As a refresher from part one of this series, the following is an image via Microsoft showing the Azure Stack TP3 architecture, click here or on the image to learn more including the names and functions of the various virtual machines (VMs) that make up Azure Stack.

Microsoft Azure Stack architecture
Click here or on the above image to view list of VMs and other services (Image via Microsoft.com)

What's Involved in Installing Azure Stack TP3?

The basic steps are as follows:

  • Read this Azure Stack blog post
  • Download the bits (e.g. the Azure Stack software) from here, where you access the Azure Stack Downloader tool.
  • Planning your deployment, making decisions on Active Directory and other items.
  • Prepare the target server (physical machine or virtual machine) that will be the Azure Stack destination.
  • Copy Azure Stack software and installer to the target server and run pre-install scripts.
  • Modify the PowerShell script file if using a VM instead of a PM
  • Run the Azure Stack CloudBuilder setup. Configure unattend.xml if needed or answer prompts.
  • Server reboots, select Azure Stack from two boot options.
  • Prepare your Azure Stack base system (time, network NICs in static or DHCP. If running on VMware, install VMtools)
  • Determine if you will be running with Azure Active Directory (AAD) or standalone Active Directory Federated Services (ADFS).
  • Update any applicable installation scripts (see notes that follow)
  • Deploy the script, then extended Azure Stack TP3 PoC as needed

Note that this is a large download of about 16GB (23GB with optional Windows Server 2016 demo ISO).

Use the AzureStackDownloader tool to download the bits (about 16GB or 23GB with optional Windows Server 2016 base image), which will either be in several separate files that you stitch back together with the MicrosoftAzureStackPOC tool, or as a large VHDX file and smaller 6.8GB ISO (Windows Server 2016). Prepare your target server system for installation once you have all the software pieces downloaded (or do the preparations while waiting for download).

Once you have the software downloaded, if it is a series of eight .bin files (7 about 2GB, 1 around 1.5GB), it's a good idea to verify their checksums, then stitch them together on your target system, or on a staging storage device or file share. Note that for the actual deployment's first phase, the large resulting cloudbuilder.vhdx file will need to reside in the C:\ root location of the server where you are installing Azure Stack.

server storageio nested azure stack tp3 vmware

Azure Stack deployment prerequisites (Microsoft) include:

  • At least 12 cores (or more), dual socket processor if possible
  • As much DRAM as possible (I used 100GB)
  • Put the operating system disk on flash SSD (SAS, SATA, NVMe) if possible, and allocate at least 200GB (more is better)
  • Four x 140GB or larger (I went with 250GB) drives (HDD or SSD) for data deployment drives
  • A single NIC or adapter (I put mine into static instead of DHCP mode)
  • Verify your physical or virtual server BIOS has VT enabled

The above image helps to set the story of what is being done. On the left is for bare metal (BM) or physical machine (PM) install of Azure Stack TP3, on the right, a nested VMware (vSphere ESXi 6.5) with virtual machine (VM) 11 approach. Note that you could also do a Hyper-V nested among other approaches. Shown in the image above common to both a BM or VM is a staging area (could be space on your system drive) where Azure Stack download occurs. If you use a separate staging area, then simply copy the individual .bin files and stitch together into the larger .VHDX, or, copy the larger .VHDX. Which is better is up to your preferences.

Note that if you use the nested approach, there are a couple of configuration (PowerShell) scripts that need to be updated. These changes are to trick the installer into thinking that it is on a PM when it checks to see if it's in a physical or virtual environment.

Also note that if using nested, make sure you have your VMware vSphere ESXi host along with specific VM properly configured (e.g. that virtualization and other features are presented to the VM). With vSphere ESXi 6.5 virtual machine type 11, nesting is night and day easier vs. earlier generations.

Something else to explain here is that you will initially start the Azure Stack install preparation using a standard Windows Server (I used a 2016 version) where the .VHDX is copied into its C:\ root. From there, you will execute some PowerShell scripts to setup some configuration files, one of which needs to be modified for nesting.

Once those prep steps are done, there is a CloudBuilder deploy script that gets run, which can be done with an unattend.xml file or manual input. This step will cause a dual-boot option to be added to your server, where you can select Azure Stack or your base prep Windows Server instance, followed by a reboot.

After the reboot occurs and you choose to boot into Azure Stack, this is the server instance that will actually run the deployment script, as well as build and launch all the VMs for the Azure Stack TP3 PoC. This is where I recommend having a rough sketch like above to annotate layers as you go to remember what layer you're working pme. Don’t worry, it becomes much easier once all is said and done.

Speaking of preparing your server, refer to Microsoft specs, however, in general, give the server as much RAM and as many cores as possible. Also. if possible, place the system disk on a flash SSD (SAS, SATA, NVMe) and make sure that it has at least 200GB, though 250 or even 300GB is better (just in case you need more space).

Additional configuration tips include allocating four data disks for Azure. If possible, make these SSDs as well as, however, I think it's more important to have at least the system on fast flash SSD. Another tip is to enable only one network card or NIC and put it into static vs. DHCP address mode to make things easier later.

Tip: If running nested, vSphere 6.5 worked the smoothest as had various issues or inconsistencies with earlier VMware versions, even with VMs that ran nested just fine.

Tip: Why run nested? Simple, I wanted to be able to use VMware tools, do snapshots to go back in time, plus share the server with some other activities until ready to give Azure Stack TP3 its own PM.

Tip: Do not connect the POC machine to the following subnets (,,,,, as Azure Stack TP3 uses those.

storageio azure stack tp3 vmware configuration

Since I decided to use a nested VM deploying using VMware, there were a few extra steps needed that I have included as tips and notes. The following is a view via vSphere client of the ESXi host and VM configuration.

The following image combines a couple of different things including:

  • A: Showing the contents of the C:\Azurestack_Supportfiles directory.

  • B: Modifying the PrepareBootFromVHD.ps1 file if deploying on virtual machine (see tips and notes).

  • C: Showing contents of the staging area, including individual .bin files along with the large CloudBuilder.vhdx.

  • D: Running the PowerShell script commands to prepare the PrepareBootFromVHD.ps1 and related items.

prepariing azure stack tp3 cloudbuilder for nested vmware deployment

From PowerShell (administrator):

# Variables
$Uri = 'https://raw.githubusercontent.com/Azure/Azure stack/master/Deployment/'
$LocalPath = 'c:\AzureStack_SupportFiles'   

# Create folder
New-Item $LocalPath -type directory   

# Download files
( 'BootMenuNoKVM.ps1', 'PrepareBootFromVHD.ps1', 'Unattend.xml', 'unattend_NoKVM.xml') | foreach { Invoke-WebRequest ($uri + $_) -OutFile ($LocalPath + '\' + $_) }

After you do the above, decide if you will be using an Unattend.xml or manual entry of items for building the Azure Stack deployment server (e.g. a Windows Server). Note that the above PowerShell script created the C:\azurestack_supportfiles folder and downloads the script files for building the cloud image using the previously downloaded Azure Stack CloudBuilder.vhdx (which should be in C:\).

A note and tip is that if you are doing a VMware or virtual machine-based deployment of TP3 PoC, you will need to change C:\PrepareBootFromVHD.ps1 in the Azure Stack support files folder. Here is a good resource on what gets changed via Github, which shows an edit on or around line 87 of PrepareBootFromVHD.ps1. If you run the PrepareBootFromVHD.ps1 script on a virtual machine, you will get an error message. The fix is relatively easy (after I found this post).

Look in PrepareBootFromVHD.ps1 for something like the following around line 87:

if ((get-disk | where {$_.isboot -eq $true}).Model -match 'Virtual Disk') {      
    Write-Host "The server is currently already booted from a virtual hard disk, to boot the server from the CloudBuilder.vhdx you will need to run this script on an Operating System that is installed on the physical disk of this server."      

You can either remove the "exit" command, or, change the test for "Virtual Disk" to something like "X", for fun I did both (and it worked).

Note that you only have to make the above and another change in a later step if you are deploying Azure Stack TP3 as a virtual machine.

Once you are ready, go ahead and launch the PrepareBootFromVHD.ps1 script, which will set the BCDBoot entry (more info here).

azure stack tp3 cloudbuilder nested vmware deployment

You will see a reboot and install. This is installing what will be called the physical instance. Note that this is really being installed on the VM system drive as a secondary boot option (e.g. Azure Stack).

azure stack tp3 dual boot option

After the reboot, log into the new Azure Stack base system and complete any configuration, including adding VMware Tools if using VMware nested. Some other things to do include making sure you have your single network adapter set to static (makes things easier), and any other updates or customizations. Before you run the next steps, you need to decide if you're going to use Azure Active Directory (AAD) or local ADFS.

Note that if you are not running on a virtual machine, simply open a PowerShell (administrator) session, and run the deploy script. Refer here for more guidance on the various options available, including a discussion on using AAD or ADFS.

Note that if you run the deployment script on a virtual machine, you will get an error that is addressed in the next section, otherwise, sit back and watch the progress.

CloudBuilder Deployment Time

Once you have your Azure Stack deployment system and environment ready, including a snapshot if on a virtual machine, launch the PowerShell deployment script. Note that you will need to have decided if deploying with Azure Active Directory (AAD) or Azure Directory Federated Services (ADFS) for standalone, AKA submarine, mode. There are also other options you can select as part of the deployment discussed in the Azure Stack tips here (a must read) and here. I chose to do a submarine mode (e.g. not connected to Public Azure and AAD) deployment.

From PowerShell (administrator):

cd C:\CloudDeployment:\Setup

$adminpass = ConvertTo-SecureString "youradminpass" -AsPlainText -Force

.\InstallAzureStackPOC.ps1 -AdminPassword $adminpass -UseADFS

Deploying on VMware Virtual Machines Tips

Here is a good tip via Gareth Jones (@garethjones294) that I found useful for updating one of the deployment script files (BareMetal_Tests.ps1 located in the C:\CloudDeployment\Roles\PhysicalMachines\Tests folder) so that it would skip the bare metal (PM) vs. VM tests. Another good resource, even though it is for TP2 and early versions of VMware, is TP2 deployment experiences by Niklas Akerlund (@vNiklas).

Note that this is a bit of a chicken and egg scenario unless you are proficient at digging into script files, since the BareMetal_Tests.ps1 file does not get unpacked until you run the CloudBuilder deployment script. If you run the script and get an error, then make the changes below and rerun the script as noted. Once you make the modification to the BareMetal_Tests.ps1 file, keep a copy in a safe place for future use.

Here are some more tips for deploying Azure Stack on VMware,

Per the tip mentioned about via Gareth Jones (tip: read Gareth's post vs. simply cutting and pasting the following, which is more of a guide):

Open the BareMetal_Tests.ps1 file in PowerShell ISE and navigate to line 376 (or in that area).

Change $false to $true and that will stop the script failing when checking to see if the Azure Stack is running inside a VM.

Next go to line 453. Change the last part of the line to read “Should Not BeLessThan 0”
This will stop the script checking for the required number of cores available.

After you make the above correction, as with any error (and fix) during Azure Stack TP3 PoC deployment, simply run the following.

cd C:\CloudDeployment\Setup
.\InstallAzureStackPOC.ps1 -rerun

starting azure stack tp3 deployment

Tip: If you are deploying Azure Stack TP3 PoC on a virtual machine, once you start the script above, copy the modified BareMetal_Tests.ps1 file.

Once the CloudBuilder deployment starts, sit back and wait. If you are using SSDs, it will take a while, if using HDDs, it will take a long while (up to hours), however, check in on it now and then to see the progress and whether there are any errors. Note that some of the common errors will occur very early in the deployment, such as the BareMetal_Tests.ps1 mentioned above.

azure stack tp3 deployment finished

Checking in periodically to see how the deployment progress is progressing, as well as what is occurring. If you have the time, watch some of the scripts, as you can see some interesting things such as the software-defined data center (SDDC), AKA software-defined data infrastructure (SDDC), AKA Azure Stack virtual environment is created. This includes virtual machine creation and population, creating the software-defined storage using storage spaces direct (S2D), virtual network and active directory along with domain controllers among others activity.

azure stack tp3 deployment progress

After Azure Stack Deployment Completes

After you see the deployment completed, you can try accessing the management portal, however, there may be some background processing still running. Here is a good tip post on connecting to Azure Stack from Microsoft using Remote Desktop (RDP) access. Use RDP from the Azure Stack deployment Windows Server and connect to a virtual machine named MAS-CON01, launch Server Manager, and for Local Server, disable Internet Explorer Enhanced Security (make sure you are on the right system, see the tip mentioned above). Disconnect from MAS-CON01 (refer to the Azure Stack architecture image above), then reconnect and launch your browser and connect to the URL (though the URL in the documentation didn't work for me).

Note the username for the Azure Stack system is AzureStack\AzureStackAdmin with a password of what you set for administrative during setup. If you get an error, verify the URLs, check your network connectivity, wait a few minutes as well as verify what server you are trying to connect from and to. Keep in mind that even if deploying on a PM or BM (e.g. non-virtual server or VM), the Azure Stack deployment TP3 PoC creates a "virtual" software-defined environment with servers, storage (Azure Stack uses Storage Spaces Direct [S2D]), and software defined network.

accessing azure stack tp3 management portal dashboard

Once able to connect to Azure Stack, you can add new services, including virtual machine image instances such as Windows (use the Server 2016 ISO that is part of Azure Stack downloads), Linux, or others. You can also go to these Microsoft resources for some first learning scenarios, using the management portals, configuring PowerShell, and troubleshooting.

What This All Means

A common question is if there is demand for private and hybrid cloud. In fact, some industry expert pundits have even said private or hybrid are dead, which is interesting. How can something be dead if it is just getting started? Likewise, it is too early to tell if Azure Stack will gain traction with various organizations, some of whom may have tried or struggled with OpenStack, among others.

Given a large number of Microsoft Windows-based servers on VMware, OpenStack, public cloud services, as well as other platforms, along with continued growing popularity of Azure, having a solution such as Azure Stack provides an attractive option for many environments. That leads to the question of whether Azure Stack is essentially a replacement for Windows Servers or Hyper-V and if only for Windows guest operating systems. At this point indeed, Windows would be an attractive and comfortable option, however, given a large number of Linux-based guests running on Hyper-V as well as Azure Public, those are also primary candidates, as are containers and other services.

software defined data infrastructures SDDI and SDDC

Some will ask that if OpenStack is struggling in many organizations and being free open source, how can Microsoft have success with Azure Stack? The answer could be that some organizations have struggled with OpenStack while others have not due to lack of commercial services and turnkey support. Having installed both OpenStack and Azure Stack (as well as VMware among others), Azure Stack, at least the TP3 PoC, is easy to install, though granted it is limited to one node, unlike the production versions. Likewise, there are easy to use appliance versions of OpenStack that are limited in scale, as well as more involved installs that unlock full functionality.

OpenStack, Azure Stack, VMware, and others have their places, alongside, or supporting containers along with other tools. In some cases, those technologies may exist in the same environment supporting different workloads, as well as accessing various public clouds, after all, hybrid is the home run for many if not most legality IT environments.

Ok, nuff said (for now…).


azure stack, cloud, hybrid cloud, microsoft azure, tutorial

Published at DZone with permission of Greg Schulz , DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}