Virtually Jason

One of my customers called me for some help troubleshooting a backup issue.  For some inexplicable reason, their VM based backup solution was failing for a bunch of VMs on a specific ESXi host.  When I got there, the first thing that I checked was the host tasks and events.  It listed a whole bunch of failed vMotion attempts for one particular VM with no VM Tools installed, so I thought that I'd take a peak at the VM console to see what I could see.  That failed, with a fairly generic message: "Unable to connect to the MKS: Connection terminated by server" 9 times out of 10, that error indicates that there's a firewall between the ESXi host and the client.  It turns out that this was 1 time out of 10, because some subsequent network troubleshooting revealed that there was nothing odd going on in that space.  My next troubleshooting step was pretty obvious; check the ESXi host logs to see if anything stood out.  So, I logged into the local console of...

Several of my customers use HP C7000 blade chassis for their ESXi hosts.  One of them asked me to help them update the firmware in their enclosures.  There are basically three different pieces of the blade chassis; each one needs updating.  The blades themselves need updating, the Interconnect Modules (basically, embedded switches) need updating and the Onboard Administrator Modules need updating.  There are different and varied steps for updating each of those components.  The blade is updated by bootstrapping with an SPP (Service Pack for Proliant) ISO.  The OA is updated through its own web interface by uploading a specific .bin file.  The Virtual Connect (which is the centralized management for all of the Interconnect Modules) is updated through its own special command line utility by uploading its own specialized .bin file. All of that is a little awkward, but it’s not really that big of a deal.  We did run into one spot of difficulty, as ...

One of my customers recently moved from standard vSwitches over to distributed vSwitches.  After performing the move, they deleted all of the standard vSwitches from the environment.  Unfortunately, a few weeks later it turned out that a few VMs had adapters that, for whatever reason, hadn't been moved over to the vDS and so I was tasked with finding any VM that might still be associated with the old (now deleted) standard vSwitches. This was an interesting challenge, as my normal approach to finding all VMs on a given vSwitch would be to simply loop through every host, get the specific vSwitch and then get all VMs on that vSwitch.  Since these vSwitches had already been deleted, that approach wouldn't work.  Eventually, I came up with this command: get-vm | % {if (($_ | get-virtualswitch).name -eq $null){$_.name}} This gets all of the VMs in the inventory.  For each VM, it then checks what vSwitch the VM is attached to.  I found that when the VM only...

This is another quickie (I've had a very PowerCLI rich couple of months!).  We've been doing a lot of mass ESXi host manipulations lately and typically grab the hosts in batches of 8.  As such, I've needed to be able to target a specific set of 8 ESXi servers with some of my PowerCLI commands.  The first 8 (named esx01-esx08) are really easy to grab with string matching: get-vmhost *esx0[1-8]* That command will return all ESXi hosts in that range.  That same trick doesn't work so well with other batches of ESXi hosts (until you get into the esx41-48 range), as the square brackets represent only a single character.  That means that get-vmhost *esx[09-16]* is not a valid construct.  Fortunately, this is a pretty easy match to make with a regular expression : esx(09|1[0-6]) That regex works because of a few important symbols.  The | is the key to it; it means "or".  A simple example would be "09|10" which matches anything with "09" or ...

We've been configuring a large number of ESXi servers that don't have local hard drives.  As such, we need to set up each system's scratch location on persistent storage, as per that KB article from VMware. That KB Article has a great section devoted to performing the process from PowerCLI and it basically takes you through the whole thing, soup to nuts, for a single ESXi host.  That's all well and good, but what if you've got 16 hosts?  Or 32?  Or 64?  Well, with a few minor changes to the procedure that they've outlined, you can very easily target every ESXi host in an environment.  Bear in mind that this process does require a host reboot, so make sure that you have your ducks in a row (and preferably do this before you place any VMs onto the cluster). Instead of their step 6, I used this command to create a folder for all ESX hosts in the inventory: get-vmhost | foreach {new-item ".$($_.name)/scratch" -itemtype directory} Instead of their s...

Here's a quickie.  As part of this big vCenter forklift, we want to verify that all VMs are communicating on the network after they've been migrated to the destination systems.  I shudder to imagine someone typing out "ping vma" and then "ping vmb" etc. for all of these machines... and so I created this command to do the work for us!  It grabs the list of all VMs from the host esx01, then does two tests.  First, it checks to see if the vm has a hostname (from VMware tools), then it attempts to ping that hostname.  If the VM doesn't have a hostname or if that VM doesn't respond to ping, the command spits out a quick message saying that the VM failed to ping. get-vmhost *esx01* | get-vm | foreach {if(!($_.guest.hostname) -or !(Test-Connection $_.guest.hostname -count 1 -quiet)){echo "$($_.name) failed to ping"}} Quick note - this command uses the test-connection cmdlet, which was introduced in PowerShell 4.0.  You can use $psversiontable....

1/26/2016 Update:  We've learned that this script doesn't elegantly handle the situation where multiple VM folders have the same name (but different parent folders).  I've put together a basic script that should help recover from that (in the comments), but I haven't had a chance to thoroughly test it.  That script is built to move all VMs from the "Discovered Virtual Machine" Folder back into their correct locations.  I'm working on an updated pair of scripts that will handle this whole situation better. 2/9/2016 Update:  I've published an updated version of the vCenter Migration Scripts  - check it out instead of these! One of my customers wanted to export their VM Folder structure and the associated permissions from one vCenter to another, in preparation for a vCenter forklift.  Once the destination vCenter was fully prepared, the goal would be to migrate ESXi hosts with VMs into the new vCenter, then move the VMs into the appropriate folders....