1. Introduction
2. Update Pipeline
   1. Packer Azure-rm Issues
3. Pipeline Configuration (YAML)
   1. Variable Groups
   2. PreBuild PowerShell Script
   3. Packer Install and azure-rm fix
   4. Packer Build
4. Testing
5. Conclusion

Introduction

So, quite a while ago I wrote a little mini-series on how to automate the deployment of AVD leveraging Azure DevOps and scripting (primarily BICEP), as part of the series I also covered how to automate Windows patching for the golden image.

I am currently revisiting my AVD automation and will be putting up some newer blogs to cover additions to the scripting and general cleanup.

---

Update Pipeline

The first thing I started to look at was my update pipeline.

I previously spoke about this in the following blog:

AVD Deployment and Update – Part 3 Automating Updates

Whilst most of the functionality will remain the same as the original blog, I will be making a major change to switch to a YAML-based pipeline. Also, there will be some small changes to allow you to target an Azure Compute Gallery in a different sub (as this is often needed) as well as allow for trusted launch machines.

Packer Azure-rm Issues

So to start with I thought I would just run my original pipeline to make sure it still worked . . . and it failed!

It looks like there is a problem with the azure-rm provider?

Hmmm, it appears that the Azure Hosted Agents (at least at time of writing) may not have the azure-rm provider installed (or a non-working version)

Thankfully the fix in this case quite easy. I simply added a command line script task to install the provider before running the build step.

This script that needs to be run is the below:

packer plugins install github.com/hashicorp/azure

The above simply installs the provider directory from the Hashicorp GitHub.

With the provider installed my old classic pipeline worked.

---

Pipeline Configuration (YAML)

The major change is that I am moving away from the old classic DevOps pipelines to use a YAML-based pipeline. This is much easier for people to use and import into their own projects, as well as being the recommended pipeline approach.

So below you will find the full YAML script for my pipeline.

variables:
  - group: HUB - Selectable Variables
  - group: HUB - Static Variables
  - group: KEYVAULT - KV-EU-MAIN
jobs:
- job: Job_1
  displayName: Image Update
  pool:
    vmImage: windows-latest
  steps:
  - checkout: self 
  - task: AzurePowerShell@5
    name: PreBuildScript
    displayName: Pre-Build PowerShell
    inputs:
      azureSubscription: 'Tighetec - Hub (REDACTED)'
      ScriptType: 'FilePath'
      ScriptPath: 'Packer/PreBuild.ps1'
      ScriptArguments: -SharedImageGalleryResourceGroup "$(SharedImageGalleryResourceGroup)" -SharedImageGalleryName "$(SharedImageGalleryName)" -SharedImageGalleryDefinitionName "$(SharedImageGalleryDefinitionName)" -SharedImageGallerySubscription "$(SharedImageGallerySubscription)"
      azurePowerShellVersion: LatestVersion 
  - task: PackerTool@0
    name: PackerInstall
    displayName: Packer Install
    inputs:
      version: '1.10.1'
  - task: CmdLine@2
    name: PackerAzureProvider
    displayName: Install Packer azure-rm provider
    inputs:
      script: >
        packer plugins install github.com/hashicorp/azure
  - task: Packer@1
    inputs:
      connectedServiceType: 'azure'
      azureSubscription: 'Tighetec - Hub (REDACTED)'
      templatePath: 'Packer/Packer.json'
      command: 'build'
      variables: |
        client_id=$(AppID)
        client_secret=$(AppSecret)
        SharedImageGalleryOldVersionName=$(PreBuildScript.oldVersion)
        location=$(vmLocation)
        SharedImageGallerySubscription=$(SharedImageGallerySubscription)
        SharedImageGalleryResourceGroup=$(SharedImageGalleryResourceGroup)
        SharedImageGalleryName=$(SharedImageGalleryName)
        SharedImageGalleryDefinitionName=$(SharedImageGalleryDefinitionName)
        SharedImageGalleryNewVersionName=$(PreBuildScript.newVersion)
        virtual_network_name=$(existingVNETName)
        virtual_network_subnet_name=$(existingSubnetName)
        virtual_network_resource_group_name=$(existingVNETResourceGroup)
        secure_boot_enabled=$(trustedLaunch)
  - task: PublishBuildArtifacts@1
    displayName: 'Publish Artifact: Completed'
    inputs:
      PathtoPublish: Packer
      ArtifactName: Completed
...

Variable Groups

At the very top, you will notice the variables property. This is needed to declare variables for use in the YAML file. You can declare variables by name and value.

These can obviously link to env or user variables if needed.

In my case, I don’t want to declare any static variables but rather link to my predefined variable groups. Luckily this is super easy. Just add the variable property and then add the – groups entry as below:

variables:
  - group: HUB - Selectable Variables
  - group: HUB - Static Variables
  - group: KEYVAULT - KV-EU-MAIN

The above links the 3 variable groups that contain my various settings one of which is an Azure Key vault (for those sensitive details) These variable groups are shared with my main AVD deployment scripts to ensure I update the right Azure Compute Gallery image.

PreBuild PowerShell Script

So next we need to sort the PreBuild PowerShell script.

This script is called in the YAML as a basic AzurePowerShell@5 task.

- task: AzurePowerShell@5
    name: PreBuildScript
    displayName: Pre-Build PowerShell
    inputs:
      azureSubscription: 'Tighetec - Hub (REDACTED)'
      ScriptType: 'FilePath'
      ScriptPath: 'Packer/PreBuild.ps1'
      ScriptArguments: -SharedImageGalleryResourceGroup "$(SharedImageGalleryResourceGroup)" -SharedImageGalleryName "$(SharedImageGalleryName)" -SharedImageGalleryDefinitionName "$(SharedImageGalleryDefinitionName)" -SharedImageGallerySubscription "$(SharedImageGallerySubscription)"
      azurePowerShellVersion: LatestVersion 

This script is responsible for grabbing the latest version of the Image from the Shared Image Gallery and also creates a few variables to be used for the version number.

Nothing too fancy as it just runs a script from my repository with a number of script arguments that are populated from the previous attached variable groups.

My numbering won’t be for everyone, but it fits my purposes. I basically number as:

1.<month>.<subversion>

So, the first patched image for January would be 1.01.0

If a second image update was run it would increment the subversion and become 1.01.1.

The PowerShell that does this magic is below:

Get-AzSubscription -SubscriptionId "$($sharedImageGallerySubscription)" | Select-AzSubscription

$sigversions = Get-AzGalleryImageVersion -ResourceGroupName $SharedImageGalleryResourceGroup -GalleryName $SharedImageGalleryName -GalleryImageDefinitionName $SharedImageGalleryDefinitionName
$latestversion = $sigversions[$sigversions.count - 1].Name

#Create new patch version number:  major.month.patch
$month = get-date -format MM
$major = $latestversion.split(".")[0]
$minor = $latestversion.split(".")[1]
$patch = [int]$latestversion.split(".")[2]

#Increment Patch version if already patched this month.
if ($minor -eq $month) {
    $patch ++
}

The script also installs the required community plugin for Windows Update, which is the part that does the grunt work to install the Windows Updates. Find the plugin here: https://github.com/rgl/packer-plugin-windows-update?tab=readme-ov-file

#Import the Windows-Update plugin into Packer
$download = 'https://github.com/rgl/packer-plugin-windows-update/releases/download/v0.15.0/packer-plugin-windows-update_v0.15.0_x5.0_windows_amd64.zip'

(New-Object System.Net.WebClient).DownloadFile($download, 'D:\packerwu.zip')

Expand-Archive -Path D:\packerwu.zip -DestinationPath $env:APPDATA\packer.d\plugins

Packer Install and azure-rm fix

The next thing to do in the pipeline is to install the relevant version of Packer onto the Hosted Agent. Add the following task to handle this install. In my case I am installing version 1.10.1.

- task: PackerTool@0
    name: PackerInstall
    displayName: Packer Install
    inputs:
      version: '1.10.1'

Now, as described before I was experiencing issues with the azure-rm provider not working correctly. So, I needed to add a CmdLine@2 command line task to install the plugin:

- task: CmdLine@2
    name: PackerAzureProvider
    displayName: Install Packer azure-rm provider
    inputs:
      script: >
        packer plugins install github.com/hashicorp/azure

Packer Build

Now onto the main event. That actual Packer build job. This task really just runs a Packer build task targeting my Packer.JSON file in my repository. I pass a number of variables such as a client_id and client_secret that is used to perform the build operations. This must have the relevant permissions over the Azure Subscription. In my case I’ve granted it contributor access, but in production you’d likely want to limit its access.

- task: Packer@1
    inputs:
      connectedServiceType: 'azure'
      azureSubscription: 'Tighetec - Hub (REDACTED)'
      templatePath: 'Packer/Packer.json'
      command: 'build'
      variables: |
        client_id=$(AppID)
        client_secret=$(AppSecret)
        SharedImageGalleryOldVersionName=$(PreBuildScript.oldVersion)
        location=$(vmLocation)
        SharedImageGallerySubscription=$(SharedImageGallerySubscription)
        SharedImageGalleryResourceGroup=$(SharedImageGalleryResourceGroup)
        SharedImageGalleryName=$(SharedImageGalleryName)
        SharedImageGalleryDefinitionName=$(SharedImageGalleryDefinitionName)
        SharedImageGalleryNewVersionName=$(PreBuildScript.newVersion)
        virtual_network_name=$(existingVNETName)
        virtual_network_subnet_name=$(existingSubnetName)
        virtual_network_resource_group_name=$(existingVNETResourceGroup)
        secure_boot_enabled=$(trustedLaunch)

The Packer.JSON file itself contains the main build logic.

{
  "variables": {

    "client_id": "",
    "client_secret": "",

    "subscription_id": "{{env `ARM_SUBSCRIPTION_ID`}}",
    "vm_size": "Standard_E8as_v4",
    "location": "",
    "secure_boot_enabled": "",
  
    "SharedImageGallerySubscription": "",
    "SharedImageGalleryResourceGroup": "",
    "SharedImageGalleryName": "",
    "SharedImageGalleryDefinitionName": "",
    "SharedImageGalleryOldVersionName": "",
    "SharedImageGalleryNewVersionName": "",

    "virtual_network_name": "",
    "virtual_network_subnet_name": "",
    "private_virtual_network_with_public_ip": "True",
    "virtual_network_resource_group_name": "",
    
    "Build_DefinitionName": "",
    "Build_BuildNumber": ""

  },
  "builders": [
    {
      "type": "azure-arm",
   
      "client_id": "{{user `client_id`}}",
      "client_secret": "{{user `client_secret`}}",
      "subscription_id": "{{user `subscription_id`}}",

      "communicator": "winrm",
      "winrm_use_ssl": "true",
      "winrm_insecure": "true",
      "winrm_timeout": "3m",
      "winrm_username": "packer",
      "os_type": "Windows",
      "secure_boot_enabled": "{{user `secure_boot_enabled`}}",


      "shared_image_gallery": {
        "subscription": "{{user `SharedImageGallerySubscription`}}",
        "resource_group": "{{user `SharedImageGalleryResourceGroup`}}",       
        "gallery_name": "{{user `SharedImageGalleryName`}}",
        "image_name": "{{user `SharedImageGalleryDefinitionName`}}",
        "image_version": "{{user `SharedImageGalleryOldVersionName`}}"
      },



      "shared_image_gallery_destination": {
        "subscription": "{{user `SharedImageGallerySubscription`}}",
        "resource_group": "{{user `SharedImageGalleryResourceGroup`}}",
        "gallery_name": "{{user `SharedImageGalleryName`}}",
        "image_name": "{{user `SharedImageGalleryDefinitionName`}}",
        "image_version": "{{user `SharedImageGalleryNewVersionName`}}",
        "replication_regions": ["North Europe"]
      },
      "shared_image_gallery_timeout": "2h5m2s",


      "temp_resource_group_name": "rg-PackerBuild",
      "virtual_network_name": "{{user `virtual_network_name`}}",
      "virtual_network_subnet_name": "{{user `virtual_network_subnet_name`}}",
      "private_virtual_network_with_public_ip": "True",
      "virtual_network_resource_group_name": "{{user `virtual_network_resource_group_name`}}",

      "azure_tags": {
        "Owner": "James Tighe",
        "Purpose": "AVD",
        "Updated": "{{timestamp}}"
      },

      "location": "{{user `location`}}",
      "vm_size": "{{user `vm_size`}}"
    }
  ],
  "provisioners": [
    {
    "type": "windows-update",
    "search_criteria": "IsInstalled=0",
    "filters": [
      "exclude:$_.Title -like '*Preview*'",
      "include:$true"
    ],
    "update_limit": 25
    },
    {
      "type":"powershell",
      "elevated_user": "packer",
      "elevated_password": "{{.WinRMPassword}}",
      "inline": [
          "New-ItemProperty -Path \"HKLM:\\SOFTWARE\\Policies\\Microsoft\\Windows\\WindowsUpdate\\AU\" -Name 'NoAutoUpdate' -Value '1' -PropertyType DWORD -Force | Out-Null",
          "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml ){ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}",
          "& $env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit /mode:vm",
          "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 10  } else { break } }"
      ]
    }
  ]
}

At the top of file, I declare a few variables that will be used later in the script. Most of these variables are populated from the variable groups. You could hardcode them if you want but I wanted the flexibility to run this against multiple environments.

Under the main builders section, we are using the azure-rm provider. This has some required settings to control the winrm settings and the client_id used to perform the build operation.

It is in this section I have added a variable to control the use of Trusted Launch machines. The variable secure_boot_enabled must be set if the Image Gallery image is a Gen 2 Secure Boot image.

"type": "azure-arm",
   
      "client_id": "{{user `client_id`}}",
      "client_secret": "{{user `client_secret`}}",
      "subscription_id": "{{user `subscription_id`}}",

      "communicator": "winrm",
      "winrm_use_ssl": "true",
      "winrm_insecure": "true",
      "winrm_timeout": "3m",
      "winrm_username": "packer",
      "os_type": "Windows",
      "secure_boot_enabled": "{{user `secure_boot_enabled`}}",

In order that I can use this same pipeline with both Secure Boot and Non-Secure Boot, I have set this to be populated from an existing variable group variable that I used in the AVD deployment itself. This means the update pipeline will run with the same settings.

We next specify a shared_image_gallery block to point to the source image for update. This is quite self-explanatory. I have however added the ability to target a Shared Image Gallery in a different subscription. This is key as if using the Landing Zone architecture for Azure you may very well have your images in the core subscription not your AVD one.

"shared_image_gallery": {

        "subscription": "{{user `SharedImageGallerySubscription`}}",

        "resource_group": "{{user `SharedImageGalleryResourceGroup`}}",       

        "gallery_name": "{{user `SharedImageGalleryName`}}",

        "image_name": "{{user `SharedImageGalleryDefinitionName`}}",

        "image_version": "{{user `SharedImageGalleryOldVersionName`}}"

      },



      "shared_image_gallery_destination": {

        "subscription": "{{user `SharedImageGallerySubscription`}}",

        "resource_group": "{{user `SharedImageGalleryResourceGroup`}}",

        "gallery_name": "{{user `SharedImageGalleryName`}}",

        "image_name": "{{user `SharedImageGalleryDefinitionName`}}",

        "image_version": "{{user `SharedImageGalleryNewVersionName`}}",

        "replication_regions": ["North Europe"]

      },

      "shared_image_gallery_timeout": "2h5m2s",

The shared_image_gallery_destination block is exactly what it sounds like. It tells Packer where the completed image should be stored. You can set replication to include multiple regions if needed. For this example, I am only replicating to North Europe.

In my previous blog I was using some managed image variable. This was basically making Packer build a managed imaged as well as saving to the Shared Image Gallery.

Due to Secure Boot Machine not being supported for Managed Images this was removed as it didn’t server any real purpose.

A few other azure-rm variables are set. Such as temp machines settings for the Azure VM, tags and VM size and location. Pretty straight-forward.

     "temp_resource_group_name": "rg-PackerBuild",

      "virtual_network_name": "{{user `virtual_network_name`}}",

      "virtual_network_subnet_name": "{{user `virtual_network_subnet_name`}}",

      "private_virtual_network_with_public_ip": "True",

      "virtual_network_resource_group_name": "{{user `virtual_network_resource_group_name`}}",



      "azure_tags": {

        "Owner": "James Tighe",

        "Purpose": "WVD",

        "Updated": "{{timestamp}}"

      },



      "location": "{{user `location`}}",

      "vm_size": "{{user `vm_size`}}"

The final bit of the Packer script is the provisioners block. This is used to run the Windows Update plugin and then run pre-imaging scripts.

"provisioners": [

    {

    "type": "windows-update",

    "search_criteria": "IsInstalled=0",

    "filters": [

      "exclude:$_.Title -like '*Preview*'",

      "include:$true"

    ],

    "update_limit": 25

    },

    {

      "type":"powershell",

      "elevated_user": "packer",

      "elevated_password": "{{.WinRMPassword}}",

      "inline": [

          "New-ItemProperty -Path \"HKLM:\\SOFTWARE\\Policies\\Microsoft\\Windows\\WindowsUpdate\\AU\" -Name 'NoAutoUpdate' -Value '1' -PropertyType DWORD -Force | Out-Null",

          "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml ){ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}",

          "& $env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit /mode:vm",

          "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 10  } else { break } }"

      ]

The first part of this block (highlighted in red) is used to call the windows-update provider with the relevant settings. In my case I am checking for any Uninstalled patched (IsInstalled=0) and excluding any preview patches (exclude:$_.Title -like “*Preview*’)

The last part runs a PowerShell task. This is set to use the packer user that is created as part of the VM creation. The task runs an inline PowerShell script thats disables Windows Update Automatic Update via the NoAutoUpdate registry value.

It also performs a few tasks to prepare the VM for imaging by removing any unattend.xml files and running a Sysprep.exe command and the waiting for the machine to change state to IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE before the task completes.

And with that the pipeline is complete!

Testing

So, now we have the pipeline all completed in nice shiny YAML rather than that dirty classic (yuck), we should be able to give it a run!

And now to wait!

The pipeline can take a while to run, as it need to first create the temporary Azure VM, then install the Windows Updates (which can also take quite some time)

Please be aware the creation of the Shared Image Gallery version can take a considerable amount of time depending on the replication.

This can cause the pipeline job to exceed the 1-hour restriction for Azure Hosted Agents. 

If this is the experience this issue, you may need to switch to using a Self-Hosted Agent as this has no such time restrictions.

Once the pipeline is complete, we should have a nice new Shared Image Gallery version ready to be deployed!

Conclusion

So, there you have it, automated Windows patching for your AVD image using DevOps and Packer.

This is a great way to automate your patching and I have to shoutout Rui Lopes for his work on the windows-update plugin.

Hopefully this might be useful to some people and help them with patch management of their AVD image.

Until next time!