Updating VM Templates with vRealize Orchestrator, Part 1

Jeff and I have been doing a lot of work with Orchestrator lately.  It's been a bit of an uphill struggle, but we've managed to create a few cool things!  This one is very simple, but it addresses a common pain point and so we're sharing it.  We've generated an Orchestrator workflow that automatically applies Windows Updates to templates.

Admittedly, that's not a very difficult process to go through by hand... but it's arduous and needs to be done regularly... and, truth be told, it seems like most people just don't bother.  So, we decided to automate it!  It turns out that automating it was actually a fairly interesting process, as we had to deal with a lot of decisions that are very easy for a human to make but are not so straight-forward for a machine.  I'm going to try my hand at a 3 part series of posts, detailing this workflow.

So, what does this Workflow look like?  Here you go:



As you can see, the workflow has 4 major parts.  The first part does whatever manipulations it needs in order to get a powered-on VM.  The second section does a bunch of preparatory steps to get that VM ready to apply those updates.  The third section is the where the magic happens, applying the windows updates until there are no more to apply.  The fourth section performs clean-up operations by removing the scripts from the system and turning the VM back into a template.

This workflow is designed to be executed against a single Template (or VM); Jeff has written a wrapper workflow that iterates through a whole list of Templates, invoking this workflow against each of them in parallel.  This workflow also depends on a custom workflow that Jeff created: Put Script on Guest.  At its core, that workflow generates a script file based on a variable within Orchestrator and uploads that file into a temp directory on the Guest OS.

We have bundled the workflows together into an Orchestrator package.  I believe that you can download this package and import it, to see what we've been doing.  This is still in development and so is posted for educational purposes.  It has worked in my limited testing, but there's no guarantee that it will work for you.  Please, take a look at it and if you come up with any improvements, let the rest of us know!

Download the Update Templates Orchestrator Package

Part 2: Update Templates Workflow Preparation Phases
Part 3: Update Templates Workflow Execution/Cleanup Phases

Comments

  1. rwedel80July 10, 2015 at 9:15 AM

    Where are the powershell scripts stored on the vRO appliance? I would like to modify them for our environment. Thanks for the great workflow!

    ReplyDelete
    Replies
    1. Jason ColemanJuly 10, 2015 at 9:46 AM

      We don't actually store them on the vRO appliance beyond what it takes to transfer them onto the Guest. Instead, we created a series of Workflow parameters (DownloadScript, ApplyScript, EnableScript and DisableScript) that each contain the contents of the appropriate script. vRO uses those parameters to generate the scripts as temp files which then get uploaded to the Guest.

      Delete
    2. Jason ColemanJuly 10, 2015 at 9:50 AM

      By the way - I published Part 2 this morning, which goes into some details about how those scripts are generated and what they're being used for:

      http://virtuallyjason.blogspot.com/2015/07/updating-vm-templates-with-vrealize_10.html

      Delete
    3. rwedel80July 10, 2015 at 1:15 PM

      Awesome, thanks for all your work on this, found where to edit the script. Everything is working great!

      Delete
  2. UnknownDecember 2, 2015 at 4:55 AM

    Hi Jason, great work! But i had some problems to get it running. My workflow break at the third part at Point "done". In the logs i found

    [2015-12-02 12:23:11.184] [I] Datei C:\Windows\temp\need-apply wurde nicht gefunden (Workflow:Delete file in guest / Scriptable task (item1)#9)
    [2015-12-02 12:23:11.506] [I] Datei C:\Windows\temp\done wurde nicht gefunden (Workflow:Delete file in guest / Scriptable task (item1)#9)
    [2015-12-02 12:23:11.858] [I] Datei C:\Windows\temp\ready-for-reboot wurde nicht gefunden (Workflow:Delete file in guest / Scriptable task (item1)#9)
    [2015-12-02 12:23:42.929] [I] Datei wurde nicht gefunden (Workflow:List path in guest / Scriptable task (item1)#7)
    [2015-12-02 12:23:42.953] [I] Datei wurde nicht gefunden (Workflow:List path in guest / Scriptable task (item1)#7) (Workflow:Check for file in guest / process error (item3)#1)

    So it means and i see it on the vm, some parts are missing and it´s true when i look into the vm. Need-apply is the last file and not a *.ps1

    Any idea what to do?

    ReplyDelete
    Replies
    1. Jason ColemanDecember 2, 2015 at 7:45 AM

      Those files (need-apply, done and ready-for-reboot) are just empty flag files, not scripts. The scripts create them at certain points in order to tell the Orchestrator workflow what to do next. It's really hard to pass information from the guest OS to Orchestrator; the easiest way that I found was to create files and then check for their existence.

      So, those error messages are probably benign; the workflow is designed to loop until it finds one of those files, at which point it can proceed to the next step. When you run it, does it complete a Windows Update cycle before going down the "done" path and failing, or does it go straight to that "done" path?

      When developing it, I found it useful to ensure that my VM could apply Windows Updates by manually pressing the "scan for updates" button. That's basically the mechanism that's being used here, and if it doesn't work, then the whole thing will fall apart.

      Delete
  3. UnknownDecember 2, 2015 at 11:29 PM

    Hi Jason, thanks for your answer......it goes striaght to that "done" path and no Windows Update cycle starts before. Maybe cause of over 160 updates for fresh installed W2k12R2? Ok i try to update the first f.e. 100 manual and then see what happens....

    ReplyDelete
    Replies
    1. Jason ColemanDecember 3, 2015 at 7:37 AM

      It might be worth investigating the PowerShell script that applies the windows updates. You can either grab it from the temp folder on the VM while the workflow is running, or just copy the text out of the variable in the workflow. Unfortunately, I don't have Orchestrator in front of me and so I can't open the workflow to figure out which variable it is at the moment though...

      Delete
  4. UnknownMay 10, 2016 at 11:14 AM

    Much appreciated with your workflow. However, I've encountered some reliability problems on some of the parts.

    I found that during the time when the workflow goes into a check that checks for need-apply/done/ready-for-reboot file existence, which fires another built-in workload "check for file in guest", can cause the whole workflow to fail with the error message in the log: The operation is not allowed in the current state. It happens right when the "process error" portion of "Check for file in guest" workload was running. This seems to happen more often than I feel like it should be, and unfortunately, it causes the template update workflow to fail.

    Should I consider increasing the sleep timer variable, or modify the sleep timer script to double the amount of the variable for the sleep time? Seems like the only way to work around this is to ensure it doesn't try to fire a "check file" workflow too many times.

    Thanks in advance

    ReplyDelete
  5. UnknownAugust 12, 2017 at 9:13 AM

    Hi Jason - do you have an updated version of this I get errors when trying to import in vRO 7.3 and vCenter 6.5. Also... silly q... how do I actually run it/configure which templates to run it again?

    ReplyDelete
    Replies
    1. Jason ColemanAugust 12, 2017 at 11:45 AM

      I don't, sorry. I haven't had a good chance to work with vRO since we put this together. We ended up running it by invoking the workflow through the vCenter Web Client by Right Clicking on a template VM.

      Delete
  6. AdrianNovember 13, 2018 at 8:41 PM

    The link is 404'd :(

    http://virtualdatacave.com/wp-content/uploads/2015/07/UpdateTemplates.zip

    ReplyDelete
    Replies
    1. JustusJanuary 4, 2019 at 8:29 PM

      Seeing the same thing. The website says it's going through some much needed maintenance. Any idea when this will be available again?

      Delete
    2. Jason ColemanJanuary 5, 2019 at 3:29 PM

      I asked Jeff about it and he's not sure what the timeframe is for getting it back online. That said, this workflow is something that we created over 3 years ago, held together with chewing gum and good intentions, so I'd be surprised if it even worked on current versions of Orchestrator.

      Delete
  7. UnknownSeptember 26, 2019 at 1:02 PM

    Hi Jason,

    Is there a way to do WSUS patching post VM provisioning and get the updates of all patches installed with a status?

    Please help me with a workflow.

    ReplyDelete
    Replies
    1. Jason ColemanSeptember 26, 2019 at 1:58 PM

      Sorry, but I haven't had much opportunity to work on Orchestrator in the past few years, so my skills there are pretty rusty. If I were to do it again at this point, I'd try to leverage SCCM and not treat the VMs as anything special, so that they could be treated as a regular part of the Windows inventory.

      Delete
  8. Kirk Van SlykeApril 7, 2020 at 8:40 AM

    Hi Jason,
    I know this blog series was from several years ago, but it is spot on with what I'm hoping to do. Unfortunately, the download link appears to be a dead end. Would it be possible to re-post the package? If not, I'm unclear about your workflow elements that copy up the script files. You said that the PS scripts are stored as parameters in the workflow until the workflow generates and uploads the script files. Can you explain that process a bit more?

    Thanks for the great blog!

    ReplyDelete
    Replies
    1. Jason ColemanApril 7, 2020 at 9:53 AM

      I'm sorry, I haven't done much work with Orchestrator since this was put together and doubt that it would work on modern versions of the environment (which is why Jeff took down his blog). I don't entirely remember how we generated those script files, but I believe that Orchestrator has a function where it can create files in the VM. So, we wrote out the script in notepad++ and then copied the whole thing into the "body" field of that function to put the files onto the systems. This is a really hazy memory though, so I could be confusing it with something else.

      Delete

Post a Comment

Sorry guys, I've been getting a lot of spam recently, so I've had to turn on comment moderation. I'll do my best to moderate them swiftly after they're submitted,

Popular posts from this blog

Clone a Standard vSwitch from one ESXi Host to Another

One of my customers is standardizing the configurations on their HP C7000 enclosures (they've been set up at various sites by various administrators with varying involvement from the architecture team).  As such, we need to stand up some temporary resources so that we can take down the main enclosure for reconfiguration.  That's fine, we can easily ship a smaller enclosure to each site for temporary compute resources.  Some of the sites are using Standard vSwitches, so we need to be able to quickly copy the networking configuration over from their existing blades to these new, slightly different blades.  As I see it, we had 2 good options: 1) Capture a Host Profile with the desired vSwitch configuration.  Delete every other component of the Host Profile so that only the networking section is applied; design the new ESXi hosts so that the vSwitches'll work with the old vmnic-to-vswitch configurations. 2) Write a script to clone the vSwitch from ones ESXi host...
Read more

PowerShell Sorting by Multiple Columns

PowerShell's Sort-Object cmdlet is super useful, especially when preparing output for human consumption.  A few people have found my blog while looking for more information about its use, specifically while looking for how to sort by multiple columns (well, properties, technically).  I've never done so (much less written about it), so I hope those folks found answers elsewhere.  But, it got me curious... and it turns out that it's really easy. The Technet article on Sort-Object has the answer directly spelled out: just use commas to create a list of properties to sort by (in order to precedence).  Let's look at some examples!  First, prepare a variable with some good sortable data: $myData = get-eventlog System -newest 25 And then we can get to sorting!  Say you want to sort primarily by EntryType (Warning, Information, Error) and then by Source.  Easy-peasy: $myData | sort EntryType,Source How about if you want it to be in descending or...
Read more

Deleting Orphaned (AKA Zombie) VMDK Files

A problem that most Virtualization Administrators need to deal with is Orphaned VMDK files (or Zombie VMDK files, as some people call them).  These are Virtual Machine hard drive files that are just hanging out, taking up storage, but are attached to no VMs.  How can this happen?  Well, the most common way is when an administrator needs to delete a VMDK but isn't quite ready to commit... so, after pressing "remove", they select "remove from virtual machine" rather than "remove and delete files".  The intent is always good: "I want to be able to restore this if it turns out to be necessary... so I'll just come back and delete it next week if nothing breaks!"  But, of course, next week something else happens and the file doesn't get deleted.  After a little while, it turns into "which files did I rename, again?" and nothing seems to happen. So, how do you deal with these orphaned files?  Well, a few years ago I found a scrip...
Read more