Virtually Jason

Hey everyone - one of my customers was doing some maintenance on the FC switches recently and wanted an easy way to ensure that all of their ESXi hosts had 4 Active links to their datastores before bringing a switch down for maintenance.  Sure, we can click through the GUI and manually examine each host... or we can automat it! So, I wrote a script to report the status of all paths to a given datastore (or datastores via wildcard matching) for all hosts that are connected to it!  The usage on this guy is pretty straight-forward: .\report-datastorePaths.ps1 -datastoreName <name of datastore to examine; multiple datastores may be specified with wildcards> -alertThreshold <if the number of active paths is less than this number, alert the user> That -alertTreshold parameter (which defaults to 4) is used to cause the script to write to the host if it ever finds a host with fewer than that number Active links.  So, if it doesn't spit out any red text, you're p...

I've been working on a script for a customer recently, and part of that script checks all of the VMs in their environment to ensure that they comply with the customer's VM Tagging policy.  As you might expect, this script revolves around the get-tagAssignment cmdlet and I've noticed an interesting quirk about how it runs!  I'm not clear on the "why" of it, but run these two commands and compare the results: measure-command {get-vm | get-tagAssignment} vs. measure-command {get-tagAssignment -entity (get-vm)} It's a profound difference!  In this customer's environment, piping the results of get-vm into get-tagAssignment takes over 73 seconds to run... but when I run get-vm as the entity for get-tagAssignment, it completes in a bit over 0.5 seconds! I'm not sure what's going on here, but I do know that once I changed my script to use the -entity property it sure ran faster!

Hey everyone - I recently put together a script to make it easier to move a VM around between various networks.  It moves the VM onto the specified Port Group and then uses VMTools to run a PowerShell script in the guest (this is for Windows VMs only, sorry!) to update its IP/DNS/Gateway/etc.  Since my old Gist'ing technique isn't working any more, so it's below as simple text (and it's on my GitHub if you want to grab it that way). But first, let's break it down a little bit!  The bulk of this script is really straight-forward.  The most interesting bit is using the PowerCLI Invoke-VMScript cmdlet to do the work inside the target VM's OS (since this operation will break network connectivity).  Invoke-VMScript is, on the surface, similar to the core PowerShell cmdlet Invoke-Command... but they way that they operate is completely different under the covers!  The practical difference that made the most impact for me is that Invoke-Command allows access to th...

I do a lot of work with PowerCLI and my favorite text editor is Notepad++ .  Recently, I've been adding more modules to my repertoire ( PowerNSX and PowerVRNI , I'm looking at you!) and I've finally decided that I need to add all of these modules' cmdlets to my syntax highlighting in NPP.  The proper way to do this is to define a custom language and add everything to it.  This can be done through the GUI (yuk) or by editing a custom language xml file. Unfortunately, there's no way to clone an existing language as a start for a custom language (or at least, I couldn't find a way to do it).  Even more unfortunately, the custom language definitions file syntax is pretty different from the inbuilt ones, so you can't just copy and then edit the XML from the already existing PowerShell language. Not if you want to do it the proper way.  It looks to me like I can trivially edit my default langs.xml file (where PowerShell is defined) and add an arbitrary set o...

A customer recently came to me with the need to use ESXCLI to configure a bunch of storage settings on all of their ESXi hosts.  He had been planning on connecting to the local console of each host and then executing the command, but wanted to know if there was a better way.  Of course there is!  We could use plink to run a script on his workstation that would establish SSH connections to his servers and then execute the ESXCLI commands... or we can do it all through PowerCLI! I actually tackled this same problem 5 years ago using get-esxcli.  At that point, I did it with the normal V1 version of the cmdlet, which required carefully spaced lists of parameters, leading to ugly lines like this:  $esxcli.storage.core.device.set($thisLUN.CanonicalName,$null,$null,$queueFullSample,$queueFullThreshold,$null) No more!  Now, we have access to get-esxcli -v2 , which is much easier to use!  Instead of needing to put a bunch of $nulls into the .set() method ...

One of my customers is considering moving some of their infrastructure around and wanted to get an idea about how their WAN connection might be impacted by the move.  They didn't have vROPS and we didn't want to enable greater vCenter logging due to space constraints on the SQL server (that tells you that we're working with some older systems, doesn't it!).  So, I decided that our best course of action would be to write a script that could run on an interval, collecting and summarizing the real-time statistics that we actually needed.  Hence the creation of summarize-VMNetUsage.ps1 ! This is a pretty straightforward script.  If you run it without any parameters, it will find the highest 20 second Average Network Usage stats from all VMs in an environment, then return a summary of its findings: VM Count, sum, average, maximum, minimum, and a date-stamp.  Then, the script enters a holding pattern until 1 hour has passed and it starts the process again.  It ...

One of my customers was doing some work on their storage network and so wanted a list of all ESXi hosts and their WWNs.  This struck me as another excellent scripting opportunity, so I pulled out my old Brocade 1:1 Zoning script and went to work.  I really only needed a one-liner for this situation, so I extracted my needed syntax from that script and quickly put it together.  Here's the (admittedly, ugly) command that I came up with: get-vmhost | select name,@{N="WWNs";E={($_.ExtensionData.config.storagedevice.hostbusadapter.PortWorldWideName | % {("{0:X}"-f$_ -split '(..)' | ? {$_}) -join ':'}) -join ", "}} That'll spit out a list with two columns, ESXi host names in one, and a comma separated list of WWNs in the other.  So, what's that ugly expression in that select statement doing?  Well, it's collecting all of the port WWNs on that ESXi host, then converting each one from decimal into hexadecimal (the "{0:X}...

I've recently had the opportunity to do some work with NSX during some server migrations.  That work has been fairly repetitive for each group of servers that we're migrating so, you guessed it, I looked for an easy scripting/automation solution, and PowerNSX proved to be the right tool for the job! As the name implies, PowerNSX is the NSX portion of PowerCLI.  Installing it is a breeze - just use a relatively up-to-date PowerShell window and fire off install-module PowerNSX .  That'll give you access to a whole set of NSX-oriented PowerShell cmdlets.  It's still in beta, so there's a lot of rough edges, but it's still proven to be an excellent tool for many tasks, including manipulating large numbers of VM Security Group Membership settings and creating Firewall rules. One of those "rough edges" involves cached instances of objects.  For example, if I want to add a bunch of VMs to various security groups based on a CSV input, I'd often write ...

This is probably one of those articles that's only going to apply to a tiny percentage of people within an already miniscule niche subset of a small population... but I'm proud of my work and so am going to post it here anyway.  One of my customers needs to move a bunch of VMs off of one SAN onto another.  Storage vMotion for the win, end of story, right?  Yes* * 99% of the problem is absolutely solved with Storage vMotion, but I'm not in the business of leaving 1% unfinished.  In this case, that 1% was a bunch of older SQL Servers, set up in 2 node pairs using Microsoft Clustering Services via shared Physical Mode RDMs.  Yikes. In theory, this process isn't too bad.  Just record the vital statistics about the RDMs, then detach them from the VMs, move the VMs (during an outage window, obviously), create new RDMs using the recorded data, and power everything back up.  This process depends on the new-harddisk cmdlet... but given that I'm writing abou...

I've recently been playing around with the -RunAsync parameter in some of my PowerCLI scripts, and I'm super impressed!  I'm also super late to the party; I mean, LucD was writing about it back in 2010, but still!  So, what's it do?  It speeds up tasks that don't need to be run sequentially, that's what it does. For example, if I have a list of VMs that all need to move into a new folder, I could do it like this: $folder = get - folder "New Folder" $vmNames = get - content MyList.txt foreach ($vmname in $vmNames){ get - vm $vmname | move - vm - destination $folder } And that would move one VM, then the next, then the next, etc.  Depending on the number of VMs, it could take a real  long time.  This process could take a while because, the way this script is written, the system will wait for each "move" to complete before initiating the next.  That's where -RunAsync  comes in. $folder = get - folder "New Folder" ...

One of my customers recently migrated a bunch of VMs between two vCenter servers .  When they finished, they realized that none of their VMs' custom attributes followed the VMs into the new vCenter environment; apparently those attributes are owned by the vCenter server, rather than stored in the VMX file.  Fortunately, they had an export of their VM inventory that included those custom fields, but they had no idea how to repopulate that data across their horde of VMs. You can see where this is going... and, as expected, I helped them put together a quick PowerCLI script that would set things straight.  This script takes 2 parameters as input: a CSV file that contains the data from the VM inventory (including each VM Name and columns for each of the custom attributes), and an array of which custom attributes need to be set (since that inventory export file is probably going to contain a lot of columns that aren't custom attributes). Once it's executed, the script goe...

Here's a quick PowerCLI one-liner.  I recently had to find all of the VMs in a customer's environment that had multiple IPv4 IP Addresses assigned to them.  Here's the command that I ended up using: Get-VM | Get-VMGuest | select VM, IPAddress | ? {($_.ipaddress = $_.ipaddress | ? {$_ -match '\.' -and !($_ -match '^169\.254\.')}).count -gt 1} That guy's a little dense, so lets break it down.  Get-VM gives me a list of all VMs in the environment, which is piped to Get-VMGuest which returns the VM name and all IPs associated with it (looking at the VM's extensiondata.guest.ipaddress only shows the first address).  From there, we select the VM and its IP Address and pass that to a Where clause that has some logic. That Where clause is going to return each VM that has more than 1 IPv4 address that is not an APIPA address.  It does that by finding only the IP Addresses that have a '.' in them (since IPv4 addresses use . delimiters whereas IPv...

One of my customers recently purchased a new SAN, with the goal of decommissioning the old one.  They used Storage vMotion to migrate all of their VMs over to the new SAN and adjusted all of the ESXi hosts to put their scratch space on a new LUN, and were ready to proceed. Many people, at this point, would just turn off the old SAN... and they might be ok.  Maybe.  At that point, the ESXi hosts are going to seriously freak out, because they just encountered an unexpected SAN failure... and we've all seen that sometimes, ESXi doesn't respond well to losing datastores unexpectedly. So, the more cautious people would right click on each datastore and unmount it, then turn off the old SAN.  While not as bad as just turning off the SAN, the ESXi hosts still expect those LUNs to be there (even if they're no longer mounted as datastores) and can still run into issues. Miss Manners insists that people follow the procedure detailed in KB 2004605 .  That article inc...

4/26/17 Update:  I changed this script so that it no longer uses the min/max VLAN numbers and instead discovers a list of valid VLANs based on the Port Groups that are defined on the VDS.  It then alerts if it sees any VLANs that are not in that list. One of my customers has several physical uplinks going into their ESXi hosts, each carrying different sets of VLANs.  They recently had an issue where an uplink with one set of VLANs was accidentally attached to a VDS that was configured for the other set of VLANs.  This wasn't a catastrophic issue, as the VDS didn't have port groups defined for those invalid VLANs and so any traffic was dropped into the bit bucket, but it did mean that 1 of the links going into that switch was useless. After we corrected the issue, we decided that we should audit the environment to see if this problem had occurred anywhere else but not been detected.  We decided that the best way to perform an initial scan of the environment w...

One of my customers was preparing to do some major ESXi host reconfiguration and so needed to shift VM workload from one cluster to another.  They had a challenge in that their clusters were running with different EVC modes, and they wanted to move VMs from the newer cluster to the older cluster.  "Impossible!" the strawman says, "it can't be done!" Well, yes and no.  That's absolutely correct that you can't vMotion a VM that powered up on an Ivy Bridge CPU back onto an ESXi host with a Sandy Bridge processor.  The reason for this is that the VM, during its power on operation, scans the CPU of its host for a list of CPU features that are available and begins potentially using those features, which means that it can't be moved to a processor that doesn't have those features.  The VM, in effect, inherits its host's EVC Mode for the lifespan of this power cycle.  Until the VM goes through a complete new power cycle (not a reboot from within th...

One of my customers recently came across an issue where their ESXi hosts were randomly crashing with a PSOD.  They had recently applied the latest SPP from HP and the latest ESXi 6.0 patches, and were now occasionally seeing these crashes with messages like "LINT1/NMI (motherboard nonmaskable interrupt), undiagnosed.  This may be a hardware problem..." As the PSOD implied, they had called HP support for help, but weren't making much progress.  I did some googling and found a really interesting blog post from Jason Whitelock about a recent ESXi update causing HP servers to PSOD .  He had come across the exact same issue and had tracked it down to the value of the iovDisableIR setting, which had changed in this latest ESXi update.  When he set it back to its original setting, the PSOD issue went away. As VMware explains it, Interrupt Remapping (the technology that's affected by this setting) enables more efficient IRQ routing and thus improves performance...

This is just a quickie (mainly for my own notes in the future): if you ever need an easy way to figure out which datastore is being referenced by a given naa number (like if you're troubleshooting datastore access issues and the logs all reference that ID type), you can use this command to search it out: get-datastore | ? {$_.ExtensionData.Info.Vmfs.Extent.Diskname -match "NAA NUMBER"}

I use PowerCLI a lot.  Like, when VMware said to stop using the C# client, I just started using PowerCLI instead of learning the Flash based web client.  As such, I log into many vCenter servers many times each day, and creating a VICredentialStore item for each vCenter that I use is one trick that saves me a lot of typing and therefore time. The New-VICredentialStoreItem cmdlet is super easy to use, which creates these credential store items.  Once you have an item created, those credentials get used automatically when you connect to a vCenter server, making the logon faster and easier.  To use it, just follow this syntax: New-VICredentialStoreItem -Host vCenterServer -User JColeman -Password SuperSecretPassword And there you go, next time you use connect-viserver vCenterServer , it will automatically pass JColeman  as the username and SuperSecretPassword  as the password. Of course, no one ever wants to do this.  Who in their right mind woul...

At one of my customers' sites today, I saw an error message that I've not seen before: VM MAC Conflict.  "Well, that's certainly not good," I thought, as I poked around at the error message.  To my chagrin, I could only find that error message for a single VM in the environment, and that error message wouldn't tell me with which other VM it was conflicting.  So, I could only think of one way to figure out what was going on with this conflict: look at the MAC Address assigned to every NIC on every VM in the environment, and figure out what was causing the conflict.  Easy! No, really, it was easy.  Had I done it by hand, I would certainly have driven myself crazy, but PowerCLI made it nice and easy.  I just used this command: (get-vm | get-networkadapter | ? {$_.MacAddress -eq "<offending MAC Address>").parent Lo-and-behold, it returned 2 VMs.  One was the known VM that had flagged the error and the other was a powered-off VM.  Maybe tha...

Here's a quick one-liner that I've been particularly pleased by.  I put this together when I was reconfiguring several c7000 chassis at a time and wanted to minimize the amount of typing that I was doing per host.  This is a big ugly one liner, but what it does is to create a new VMK interface on the specified Port Group, assign it a valid IP Address and enable vMotion for that interface.  The command performs those tasks on all ESXi hosts in the "unconfigured" folder. How does it get the valid IP Address?  Well, it takes the first 2 octets of the host's management IP address, uses a customized 3rd octet, then appends the last octet from the host's management IP address.  So, for example, if the host is managed at 192.168.1.101, the script will use 192.168.#.101 for the vMotion interface. In this example, it uses a port group called "vMotion" on a VDSwitch that ends with Intranet, and has a standard /24 network with 2 in the third octet for vMoti...