Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Azure Security Audits With Pester

DZone's Guide to

Azure Security Audits With Pester

Pester is a versatile testing framework built using PowerShell. In this article, we will look at using Pester to audit our Azure infrastructure's security compliance.

· Cloud Zone ·
Free Resource

Site24x7 - Full stack It Infrastructure Monitoring from the cloud. Sign up for free trial.

We’ve previously discussed using Pester to test Azure resources in our infrastructure pipeline. In that article, we used Pester to run against a deployed Azure environment to ensure what we thought we had deployed had actually been deployed. In this article, we’re going to take this a bit further and look at using Pester to validate that we have setup our environment in a secure manner and look for areas of concern — essentially a security audit.

Concept

Security is obviously a significant concern for those deploying into the cloud. Azure provides a wide array of tools and techniques to help with this, but these are only useful if they are used. We are going to write a set of tests that can be run against an Azure Resource Group and check that the resources in there are following our security standards.

There are obviously many ways to achieve this — you could use Azure Security Center, but this restricts you only to the rulesets they provide. You could use resource policies, but these are still in preview and take a lot of work to set up. What we are producing is a simple set of tests that we can run against any resource group we like, with any number of resources, and quickly see results.

If you're just looking for the full script to run, you can find it here on GitHub.

Requirements

So the first thing we need to do is decide what we want to test for. Obviously, we can test pretty much anything that is available through Azure PowerShell, but for this, we will focus on some of the most common security configurations. This is a good starter that you can then expand however you see fit.

We will test that:

  • All VMs are configured with Antivirus/Antimalware protection
  • All VMs are behind an NSG
  • All VMs have BitLocker encryption enabled on all drives
  • No NSGs have ports open for “All”
  • All storage accounts have storage service encryption enabled
  • All Azure SQL databases are using transparent encryption
  • All Azure SQL databases have threat detection turned on
  • All resource groups have Azure Security Centre enabled.

Tests

Setup

We want to make these tests fairly generic so that we can run them against any resource group and it will test all the resources in that resource group no matter how many there are. So, the only information we need to feed into our script is the resource group name.

param(

[Parameter(Mandatory)]

[ValidateNotNullOrEmpty()]

[string]$ResourceGroupName

)


Virtual Machine Tests

We’ll start by implementing the virtual machine tests. We’re going to use the Pester Describe Section to delineate the different type of resource tests, and inside the Describe block, we will set up any resources that are going to be used by all tests. So in this section, we will get all of the VMs in the resource group, so we can use these in our tests without calling get-azurermvm every time.

Describe "Virtual Machine Tests" {

$VMs = Get-AzureRmVM -ResourceGroupName $ResourceGroupName


Then we’re going to group our tests into context blocks for different types of VM tests. So the first VM test is going to be for Antivirus, and we are going to check two things here:

  1. That the Microsoft Antimalware extension is installed
  2. That real-time protection is enabled

We are just going to use regular Azure RM PowerShell commands to get the information about the available VM extensions and how they are configured, then apply this data against our expected criteria to test. We’re also going to loop through all the VMs in the resource group to make sure we test them all.

Context "Antivirus" {

    foreach ($vm in $vms) {

        $avExtension = Get-AzureRmVMExtension -ResourceGroupName $ResourceGroupName -VMName $vms[0].name -Name IaaSAntimalware
        $publicSettings = ConvertFrom-Json $avExtension.PublicSettings

        It "$($vm.name) Should Have Micrsoft Antimalware Extension Installed" {

            $avExtension | Should Not Be $null
            $avExtension.ProvisioningState | Should Be "Succeeded"
            $publicSettings.AntimalwareEnabled | Should Be "True"
        }

        It "$($vm.name) Should Have Real Time Protection Enabled" {
            $publicSettings.RealtimeProtectionEnabled | Should Be "true"
        }
    }

}


As you can see, in some of the “IT” sections, we are actually testing multiple things, and if one of those fails, the whole test will fail. We’re checking the extension is installed and enabled, and then, in our second test, checking the properties to see that real-time protection is enabled.

In our next set of tests, we want to make sure that every VM has a Network Security Group applied to it so that we can be sure no VMs are out on the internet with no security filtering of traffic. This test is a little bit more complicated, as a VM can get an NSG from two places, either applied to its network card or applied at the subnet level — so we need to test both of these and only fail if both are missing.

Context "VM Network Security Groups" {

    foreach ($vm in $vms) {

        foreach ($nicID in $vm.NetworkProfile.NetworkInterfaces) {
            $Nic = Get-AzureRMNetworkInterface -ResourceGroupName $ResourceGroupName | where { $_.Id -eq $nicID.id }
            $nicNSG = $nic.NetworkSecurityGroup
            $subnet = $nic.IpConfigurations.subnet
            $VirtualNetwork = Get-AzureRMVirtualNetwork -ResourceGroupName $ResourceGroupName | Where { $_.Subnets.ID -match $subnet.id }
            $subnetNSG = $($VirtualNetwork.Subnets | Where { $_.ID -match $subnet.id }).NetworkSecurityGroup

            It "$($vm.name) NIC $($nicID.name) Should Have an NSG Enabled" {
                ($nicNSG -eq $null) -and ($subnetNSG -eq $null)| Should Not Be $true
            }

        }

    }
}


The thing to remember here is that the actual assertion we are testing here is just comparing the two values on either side of the should statement, so we can do whatever PowerShell conversion we need to do on the left to get the aggregated result to then pass to the comparison. So here, we are testing both NIC and subnet level NSGs and combing the results to give an overall true/false to compare.

The final VM-based test we are undertaking is to check that all VMs have BitLocker encryption enabled. Fortunately, the encryption status of the VM is exposed through the storage profile, so we can get this data easily. We undertake two tests, one for the OS disk and one for the data disks, checking they all show as “Encrypted”:

Context "VM Bitlocker Encryption" {
    foreach ($vm in $vms) {
        $encryptionStatus = Get-AzureRmVMDiskEncryptionStatus -ResourceGroupName $resourcegroupname -VMName $vm.name

        It "$($vm.name) Should have an encrypted OS disk" {
            $encryptionStatus.OsVolumeEncrypted | should be "Encrypted"
        }

        It "$($vm.name) Should have encrypted Data disks" {
            $encryptionStatus.DataVolumesEncrypted | should be "Encrypted"
        }

    }

}


NSG Tests

We’ve already tested that all VMs have an NSG, but now we want to make sure that none of these NSGs have rules that allow inbound access to the “All” group. In our environments, we want to always scope NSG rules to specific IP ranges. This might not be applicable to everyone, of course. Obviously, in some environments, you do need to open ports to the whole internet, but you can tailor this as required. Again, we are going to use a Describe and Context block to group things, even though we only actually have one test at the moment.

Describe "Network Security Group Tests" {

    $NSGS = Get-AzureRmNetworkSecurityGroup -ResourceGroupName $ResourceGroupName

    Context "Ports Open to All" {
        foreach ($NSG in $NSGS) {
            $openAllCount = 0
            foreach ($rule in $NSG.SecurityRules) {
                if ($rule.Direction -eq "Inbound" -and $rule.SourceAddressPrefix -eq "*") {
                    $openAllCount ++
                }
            }
            It "$($NSG.name) Should Have no inbound rules open to all" {
                $openAllCount| Should Be 0
            }
        }

    }

}


As you can see, we are getting all the NSGs in a resource group and looping through them, and then looping through the rules in each NSG. We count how many rules allow inbound “all” and if it’s more than 0, we fail the test.

Storage Account Tests

For storage accounts, we want to check that storage account encryption is enabled. This is actually configured at the storage resource level, and we are interested in blob encryption, so we test that. Again, looping through all storage accounts in the resource group.

Describe "Storage Account Tests" {
    $storageAccounts = Get-AzureRmStorageAccount -ResourceGroupName $ResourceGroupName

    foreach ($storageAccount in $storageAccounts) {
        It "$($storageAccount.StorageAccountName ) Should have encrypted blob storage" {
            $storageAccount.Encryption.Services.Blob.enabled | should be $true
        }
    }
}


Azure SQL Tests

Azure SQL has a couple of security measures we want to check for. The first is transparent data encryption. This is where the entire DB is encrypted and the decryption of this is handled by Azure as required, meaning no application changes are required. Because of this, it is really recommended to be used wherever possible.

The second is threat detection, which enables additional auditing on the SQL database to try and detect threats (and can be fed into Azure Security Center). There is a cost to this, so it may not be appropriate for everyone, but we are going to check that is enabled. Again, there are convenient Azure RM PowerShell commands that will check this for us.

Because the top-level unit for SQL is the server, what we need to do here is get all Azure SQL servers in a resource group, loop through these, and then loop through the databases in each server. We do also have to do a check on the database name, as using get-AzureRMSqlDatabase does retrieve the master DB, but you can’t enable threat detection or encryption on this, so we only run these tests when the DB is not “master”.

Describe "Azure SQL Tests" {
    $sqlServers = Get-AzureRmSqlServer -ResourceGroupName $ResourceGroupName
    foreach ($sqlserver in $sqlServers) {
        $sqlDatabases = Get-AzureRmSqlDatabase -ServerName $sqlServer.ServerName -ResourceGroupName $ResourceGroupName

        foreach ($sqlDatabase in $sqlDatabases) {
            if ($sqlDatabase.databaseName -ne "Master") {
                $tdeStatus = Get-AzureRmSqlDatabaseTransparentDataEncryption -ServerName $sqlserver.ServerName -DatabaseName $sqlDatabase.databaseName -ResourceGroupName $ResourceGroupName
                $threatDetectionStatus = Get-AzureRmSqlDatabaseThreatDetectionPolicy -ServerName $sqlserver.ServerName -DatabaseName $sqlDatabase.databaseName -ResourceGroupName $ResourceGroupName
                It "$($sqlDatabase.DatabaseName) on server $($sqlServer.serverName) Should have TDE Enabled" {
                    $tdeStatus.State| should be "Enabled"
                }

                It "$($sqlDatabase.DatabaseName) on server $($sqlServer.serverName) Should have Threat Detection Enabled" {
                    $threatDetectionStatus.ThreatDetectionState| should be "Enabled"
                }
            }
        }
    }

}


Azure Security Center Tests

The final test we want to do is check that Azure Security Centre data collection is enabled for this resource group. ASC will automatically onboard new resources to have their data collected if you tell it to. This is set in an ASC resource policy, this can be set at the subscription or resource group level. If set at the subscription level it will apply to the resource groups unless an exception is added.

The standard Azure RM PowerShell cmdlets don’t include commands to interact with ASC, fortunately, there is a module you can download from here that will do this for you, so you will need to install that to run this example. Using this module, we collect all the policies for this subscription, then locate the one for this resource group and check that automatic log collection is enabled:

Describe "Azure Security Center Test" {
    Import-Module "Azure-Security-Center" -WarningAction SilentlyContinue
    $ascPolicies = Get-ASCPolicy
    $rgPolicy = $ascPolicies | Where-Object { $_.name -eq "$resourcegroupname"}

    It "$($resourcegroupname) should have Azure Security Center Enabled" {
        $rgPolicy.properties.logCollection| should be "On"
    }

}



Running Tests

Now we have written all the tests, all we have to do to run them is execute this ps1 file and supply a resource group to test. This assumes you have already logged in through Login-AzureRmAccount and selected the appropriate subscription. Once we execute the file, it will run through all the tests, looping as required and outputting the results, pass or fail. In the example below, I’m running this on an arbitrary demo resource group, and everything is passing except the BitLocker encryption, which I haven’t yet enabled.

It’s very clear what is failing, what is good, and where I need to take action. Running these tests takes a couple of minutes, and you can immediately see where the issues are.

Once you have this running, you could very easily have this audit run whenever you deploy an environment, as part of your overnight testing, or even on a regular interval to check for non-compliance that may have crept in (although if this is something you want to do a lot of then this may be the point to start looking at configuration management systems which can incorporate these tests).

Next Steps

Hopefully, this has given you a good introduction into using Pester for Azure auditing. In no way have we covered everything you would need in a complex audit, but it should give you a starting point.

The full script is available here – https://github.com/sam-cogan/AzureAudit. I will be looking to evolve this to add more tests and make it a useful boilerplate for people to start auditing their Azure infrastructure. If you have tests you want to add in, please do submit a pull request and I will be happy to add them.

Site24x7 - Full stack It Infrastructure Monitoring from the cloud. Sign up for free trial.

Topics:
cloud ,microsoft azure ,pester ,testing ,tutorial

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}