Updating VM Templates with vRealize Orchestrator, Part 3
In this post, I'm going to discuss the details of the second half of the Windows Template Update workflow that we created in vRealize Orchestrator. If you want to read an overview of the script, please check out Part 1! If you want to read about the first half of the workflow, check out Part 2!
The bulk of the work for this workflow happens in the "Download then Install Windows Updates until there are no more to install" section. By the time the workflow has hit this section, it has a powered on VM that has Windows Updates enabled and all of the necessary scripts in place. This section begins by executing the Download script. This workflow step completes almost immediately, even though the script will take several minutes to complete, so it goes into a sleep (as Windows Updates can a while to download everything). The final step of that script is to create a flag file - it either generates a "Done" file if there are no more updates to download or a "NeedApply" flag file if it has found updates that are not on the system.
Once Orchestrator finds the "NeedsApply" flag file, it moves on by deleting that file and then running the Apply.ps1 script. Once again, that workflow step completes almost instantly but the script can take quite a while to complete, so the workflow enters a loop while it waits for the appropriate flag file to be generated by the script. Once the "ReadyForReboot" flag file is found, the workflow proceeds. It deletes that flag file and issues a Shutdown to the VM (as that is frequently required for updates to complete their process).
I found that some rounds of updates can take a VERY long time to apply, during the shutdown process. The Shutdown workflow has a timeout built into it that I wasn't able to manipulate, so I set up a loop to catch the failure of that Shutdown command. It basically sits in that loop, checking the power state of the VM every 30 seconds, until the VM is off. At that point, it proceeds.
The workflow next turns the VM back on so that it can run through the whole cycle again, as Windows Update frequently needs multiple passes to bring a machine fully up to date. Before it goes into its next update cycle though, it checks to make sure that the repeat is allowed. I built in a hard limit to the number of iterations the workflow will run through (set as a workflow parameter). If that limit is hit (right now, it's 15 iterations), the workflow will bomb out. I figure that if the system goes through that many repetitions, it probably means that something is broken (perhaps a patch is failing to install each time) and an administrator needs to take a look at the system. Once they've resolved the issue, they can start the workflow on that machine again and it will resume performing update cycles.
During that update cycle, if the Download.ps1 script finds that it has no updates, it creates a "Done" flag file instead of the "NeedApply" flag file. The existence of the "Done" flag file causes Orchestrator to move on to the final section of the workflow: Clean-up.
In the Clean-up section, Orchestrator disables automatic Windows Updates (by the Disable.ps1 script) and then deletes all of the files that it has created. After that point, it shuts down the VM and converts it back into a template. And with that, the workflow is completed - it has successfully applied Windows Updates and fully updated that VM!
So, how do you use it? Well, it can be set up individually per template, but that's too much work. Instead, Jeff created the Update Templates workflow. That workflow is a wrapper that accepts a whole list of VM Templates as input, then fires off a bunch of instances of the Update Template workflow. Do you want to know the really cool thing about the Update Templates wrapper? All of those Update Template instances are fired off in parallel, so you don't need to wait for one VM to complete before it moves on to the next (in my testing, it was taking about 3 hours to complete all of the Windows Update cycles that were needed).
Download the Update Templates Orchestrator Package
*This package is provided for educational purposes only and is not guaranteed to work in any environment. Make sure that you carefully understand any code that you find on the internet and test it thoroughly before executing it in your environment.
Part 1: Update Templates Workflow Overview
Part 2: Update Templates Workflow Preparation Phases
The bulk of the work for this workflow happens in the "Download then Install Windows Updates until there are no more to install" section. By the time the workflow has hit this section, it has a powered on VM that has Windows Updates enabled and all of the necessary scripts in place. This section begins by executing the Download script. This workflow step completes almost immediately, even though the script will take several minutes to complete, so it goes into a sleep (as Windows Updates can a while to download everything). The final step of that script is to create a flag file - it either generates a "Done" file if there are no more updates to download or a "NeedApply" flag file if it has found updates that are not on the system.
Once Orchestrator finds the "NeedsApply" flag file, it moves on by deleting that file and then running the Apply.ps1 script. Once again, that workflow step completes almost instantly but the script can take quite a while to complete, so the workflow enters a loop while it waits for the appropriate flag file to be generated by the script. Once the "ReadyForReboot" flag file is found, the workflow proceeds. It deletes that flag file and issues a Shutdown to the VM (as that is frequently required for updates to complete their process).
I found that some rounds of updates can take a VERY long time to apply, during the shutdown process. The Shutdown workflow has a timeout built into it that I wasn't able to manipulate, so I set up a loop to catch the failure of that Shutdown command. It basically sits in that loop, checking the power state of the VM every 30 seconds, until the VM is off. At that point, it proceeds.
The workflow next turns the VM back on so that it can run through the whole cycle again, as Windows Update frequently needs multiple passes to bring a machine fully up to date. Before it goes into its next update cycle though, it checks to make sure that the repeat is allowed. I built in a hard limit to the number of iterations the workflow will run through (set as a workflow parameter). If that limit is hit (right now, it's 15 iterations), the workflow will bomb out. I figure that if the system goes through that many repetitions, it probably means that something is broken (perhaps a patch is failing to install each time) and an administrator needs to take a look at the system. Once they've resolved the issue, they can start the workflow on that machine again and it will resume performing update cycles.
During that update cycle, if the Download.ps1 script finds that it has no updates, it creates a "Done" flag file instead of the "NeedApply" flag file. The existence of the "Done" flag file causes Orchestrator to move on to the final section of the workflow: Clean-up.
In the Clean-up section, Orchestrator disables automatic Windows Updates (by the Disable.ps1 script) and then deletes all of the files that it has created. After that point, it shuts down the VM and converts it back into a template. And with that, the workflow is completed - it has successfully applied Windows Updates and fully updated that VM!
So, how do you use it? Well, it can be set up individually per template, but that's too much work. Instead, Jeff created the Update Templates workflow. That workflow is a wrapper that accepts a whole list of VM Templates as input, then fires off a bunch of instances of the Update Template workflow. Do you want to know the really cool thing about the Update Templates wrapper? All of those Update Template instances are fired off in parallel, so you don't need to wait for one VM to complete before it moves on to the next (in my testing, it was taking about 3 hours to complete all of the Windows Update cycles that were needed).
Download the Update Templates Orchestrator Package
*This package is provided for educational purposes only and is not guaranteed to work in any environment. Make sure that you carefully understand any code that you find on the internet and test it thoroughly before executing it in your environment.
Part 1: Update Templates Workflow Overview
Part 2: Update Templates Workflow Preparation Phases
I wanted to say this Workflow is great and has been something which I have been wanting to implement for awhile. So thanks for taking the time to create it.
ReplyDeleteMy question is if there were any specific PowerShell requirements? I am testing on Windows 2008R2 Datacenter edition and it doesn't seem to every download and apply the updates.
Thanks again for any insight?
Hello Jason, when I import the package I get: IOException not a valid package file, dunes-meta-inf is missing
ReplyDeleteobviusly pls ignore my previous message!! It was end of the day.......
ReplyDeleteHello again Jason, I'd like to edit the scripts that are pushed to the Windows O/S. Where can I find them?
ReplyDeleteThey're baked into the workflow as variables.
DeleteIt might useful to all to know that in order for Powershell to execute scripts in this context, the Execution Policy should be set accordingly. A remotely signed should do
ReplyDeleteHi Jasn
ReplyDeleteI am getting the error below when i try to download the script
The site configured at this address does not contain the requested file.