Virtually Jason

One of my customers recently experienced a strange issue.  One of their ESXi hosts had entered a problem state where Storage vMotion and vMotion were failing for all VMs on the host (vMotion was failing at 13%, which is an interesting spot).  We initially noticed the issue when Storage vMotion repeatedly threw an error for one of their VMs: A general system error occurred: Failed to create journal file provider: Failed to open "/var/log/vmware/journal/..." for write: There is no space left on the device. Well, that error seemed self explanatory.  I connected to the host's CLI (SSH was already enabled, which made that easier) and did a "vdf -h" to look at its file system.  I was surprised to find that none of the partitions were full, so I dug deeper. I decided to take a look at the vmkwarning log file, which is frequently a gold mine when troubleshooting ESXi host issues.  So, I did a quick "tail /var/log/vmkwarning.log" and, lo and  behold, we h...

Towards the start of the year, I made a post about my vCenter to vCenter migration process , which can be used to update from vCenter 5.5 to vCenter 6 (or for that matter, to downgrade, or whatever).  Since then, I've done a lot of these migrations and the scripts/procedure have matured.   Jeff and I have fixed many bugs, but the reason that I'm posting about this (again) is that I recently added a new feature and have polished our migration process. The latest versions of the migration scripts (the ones that get the settings from the source vCenter and then recreate those settings on the destination vCenter) now support DRS Rules!  The native support for DRS Rule manipulation through PowerCLI is a bit lacking... but fortunately, the community has solved that problem!   Matt Boren and Luc Dekens have created a module called DRSRule that is built to help with the reading and manipulation of DRS Rules (pretty self explanatory, right?).  I took advantage of t...

One of my customers uses DRS groups heavily.  The environment contains a set of specialized ESXi servers for a specific subset of machines and a set of general ESXi servers for everything else.  However, in order to maximize availability, all of these ESXi servers are part of one big cluster.  That means that every VM in their environment must be a member of at least one DRS group, placing the VM on either these specialized ESXi servers or the general ESXi servers.  Until we get an automated solution in place (probably through Orchestrator), we need to depend on the administrators to place the VM in the correct group (almost always the general group) upon machine creation.  As you might expect, that's not going too well. So, I put together a quick PowerCLI script to detect any VMs that are not grouped and (optionally) place them into the specified group.  This is what I came up with! The script is easy to use, but it requires a non-standard PowerCLI mod...

Last year, I wrote a post with a quick one-liner about how to ping all VMs on a given ESXi host .  Since then, I've been doing a lot of work with vSphere 6 upgrades, which has involved migrating many VMs between various switches.  As part of that process, I like to have an established basic test that I can run before and after the migration, so that I can record that the migration was successful. So, needless to say, I've expanded on that old one-liner a little bit... and now it's a full blown script.  I've built this script based around my migration validation use case, and so it has some very specific behaviors based around that.  It accepts four parameters, but really only two of them are required: -vmHost and -results.  As you certainly expect, -vmHost is the name of the host on which you want to ping all VMs and -results is a path to a file where the script will store its results (I just use a .txt file, myself). When the script is executed for the firs...

I often find myself working with CSV files when I'm writing PowerShell or PowerCLI scripts.  Of course, PowerShell has the native Import-CSV cmdlet , which works well... but it doesn't have much error checking.  After writing a bunch of script specific error checking, I've finally broken down and put together a function to do it for me.  Now, I can either add this function to my PowerShell profile, or just include it in any scripts that need it. The function is called Import-ValidCSV.  It basically just calls the normal Import-CSV cmdlet, but accepts a second arguement: -RequiredColumns.  The -ReqiredColumns argument is set up to accept an array of strings, listing all of the column headers that are required (duh!).  The function iterates through the list of RequiredColumns and checks that each column exists in the supplied CSV file.  If a column does not exist, it throws an error and quits.  If all columns exist (really, just if it doesn't thro...

One of my customers recently had a catastrophic thermal event in one of their datacenters and so had to shut down all of their infrastructure at that site.  After the cooling issue was resolved, we were asked to help them to get their infrastructure back online.  Fortunately, we have included several small details as best practices in our vSphere designs, and one of those really paid off for us.  We always create a VM to Host affinity rule that keeps one Domain Controller, the vCenter server, its PSC and its Database (if external) on a known host in the management cluster. So, after the SAN was powered back on and we restarted the physical ESXi servers, we knew exactly what to do.  I fired up the vSphere client and logged into that ESXi server in the management cluster as root.  From there, I was able to easily find those core infrastructure VMs and powered them all on.  Once they were running, I logged into vCenter... and found that I had an interesting ...

I was recently deploying a series of VMware vCenter Server Appliances for a customer who wanted to migrate to that platform from their windows based vCenter 5.5 environment.  Rather than deploying all of these by hand, we figured that this was an excellent time to check out the VCSA command line install options. The first thing that we had to do was to figure out our architecture.  In this case, it was pretty easy.  Each site was getting a vCenter appliance and a PSC appliance.  We decided that we wanted our PSC appliances to replicate with each other, so that we could use the enhanced Linked Mode functionality in vSphere 6, and so designed a ring topology to reduce the impact of a given site being offline. The first thing that we had to do was to prepare some JSON answer files.  To do that, open your vCenter Server Appliance install ISO and browse to vcsa-cli-installer\templates\install and examine the bounty contained within. Copy the example .JSO...