Virtually Jason

A customer recently came to me with the need to use ESXCLI to configure a bunch of storage settings on all of their ESXi hosts.  He had been planning on connecting to the local console of each host and then executing the command, but wanted to know if there was a better way.  Of course there is!  We could use plink to run a script on his workstation that would establish SSH connections to his servers and then execute the ESXCLI commands... or we can do it all through PowerCLI!

I actually tackled this same problem 5 years ago using get-esxcli.  At that point, I did it with the normal V1 version of the cmdlet, which required carefully spaced lists of parameters, leading to ugly lines like this: $esxcli.storage.core.device.set($thisLUN.CanonicalName,$null,$null,$queueFullSample,$queueFullThreshold,$null)

No more!  Now, we have access to get-esxcli -v2, which is much easier to use!  Instead of needing to put a bunch of $nulls into the .set() method to space out our values, we can use the …

I recently worked with a customer to upgrade their Horizon VDI environment's Windows vCenter 5.5 server to the vCenter Server Appliance running 6.5.  I knew from an earlier experience that such a migration could potentially be challenging, but I hoped that things would go more smoothly this time, since that old issue was from before the Migrate option was introduced.  This customer also had a smaller, completely isolated DR VDI environment that we could upgrade first, to prove out our process.  So, that's what we did!

The migration of the DR environment went without a hitch.  We even spun up about 20 desktops and had a few IT staff log into and use them during the upgrade, so that we could be confident that we'd identify any issues that might impact the users during the production migration.  Everything went great, so we confidently moved forward with the production migration.  You can probably guess what happened next.

Fortunately, we didn't run into any catastrophic …

I've written before about creating NSX Distributed Firewall Rules following a model that uses rules that will specifically hit traffic based on if it's Inbound or Outbound.  That model is also useful for creating NSX Security Policies, as there's no negative logic (NOT applied to object) in the rule set.  While that model works great, it can be a bit difficult to wrap your head around.  In turn, that can make it difficult to hand off to a customer... so we've been working on an alternate model.

Unfortunately, this model does not work with Service Composer Policies, but it's flexible enough that it doesn't really need them.  It's based on a set of generic Security Tags (with corresponding Security Groups), that interact to create a dynamic micro-segmentation solution.  This model is based on defining a set of DFW Sections, each of which serves a very specific purpose in blocking or allowing traffic.  When creating new firewall rules, the administrator only n…

One of my customers is considering moving some of their infrastructure around and wanted to get an idea about how their WAN connection might be impacted by the move.  They didn't have vROPS and we didn't want to enable greater vCenter logging due to space constraints on the SQL server (that tells you that we're working with some older systems, doesn't it!).  So, I decided that our best course of action would be to write a script that could run on an interval, collecting and summarizing the real-time statistics that we actually needed.  Hence the creation of summarize-VMNetUsage.ps1!

This is a pretty straightforward script.  If you run it without any parameters, it will find the highest 20 second Average Network Usage stats from all VMs in an environment, then return a summary of its findings: VM Count, sum, average, maximum, minimum, and a date-stamp.  Then, the script enters a holding pattern until 1 hour has passed and it starts the process again.  It does this for 2…

Well, that title sure is a mouthful!  But, it's also what this post is all about, so let's get to it!  One of my customers was experiencing an issue where it was taking longer than expected for an NSX firewall rule publish to propagate to all of their ESXi hosts. While troubleshooting the core issue, they needed a way to get better visibility into the process so that they'd know when their publishes had succeeded.  That data was not available in the GUI, but after asking a few friends at VMware, we learned that we could get to it through the API by a simple command: GET /api/4.0/firewall/globalroot-0/status.  Those are the facts that we collected, so here's what we did with them!

First, I knew that one of my customers had done some work with the NSX API, so I asked him for some advice.  He pointed me at one of Mark Wahl's articles and gave me an excellent framework to build on.

I used that NSX API framework to send the GET command that we'd collected, which gav…

One of my customers is organizing a cloud migration and asked for help with the onboarding process.  My team and I started doing research and we come across VMware's Hybrid Cloud Extension (HCX) technology.  It's incredible, how did I not know about this before!?

The long and short of it is that it bridges customer networks into cloud datacenters so that VMs can be vMotioned to and from the cloud.  That's a very powerful position to put the customer in, as they can now migrate workload dynamically onto the cloud without taking a service outage.  How's it work?

HCX requires several appliances, both in the cloud and client datacenters.  Those appliances serve 2 major functions: they bridge production networks and they proxy ESXi hosts.

As far as network bridging is concerned, the HCX appliances function very much like an NSX Edge that is doing its own L2 bridging.  From a network perspective, HCX basically looks like an upstream switch, behind which are a series of IP an…

I've been helping one of my customers set up a proof of concept NSX implementation, which has involved configuring and then destroying several firewall designs.  In order to speed up this process, we've had to get pretty good at using PowerNSX to script out the creation of those NSX firewall rules (and other security objects).

First, how do you get PowerNSX?  Just like PowerCLI!  Open up your PowerShell window, then use this command: Install-Module PowerNSX

Now that you've got PowerNSX installed, take a moment to look at what it does for you.  Look at all of the available cmdlets by using: get-command -module PowerNSX

There's a lot going on there!  In general, the PowerNSX cmdlets use the normal PowerShell verbs: get, set, add, remove, and new, and the nouns are prefixed with NSX.  So, if you're using tab completion to figure out what command you're doing, <verb>-nsx... is usually a pretty safe place to start.  For example, if I want to get my security ta…

One of my customers recently asked me to generate a report showing all of the VMs that they had created within the last 12 months (ideally, broken down by OS), and then another showing the same for 24-12 months ago.  I did a bunch of digging around and couldn't find any attribute on the VMs that showed their creation date.  Some research revealed that the standard solution to this problem is to get-vievents for all of the VMs, then look at the date of the first event.

Unfortunately, this customer had performed a vCenter migration about a year ago, so our logs weren't intact for this purpose.  I was stumped, but one of the other admins came up with a good idea: look at the AD objects instead of the VM objects.  AD objects actually have a .whenCreated attribute, so we just need to grab them all and then find the ones for our desired timeframes.

Of course, that approach grabs all AD computers, including desktops.  We just needed a list of servers (we knew that all servers would b…

One of my customers was doing some work on their storage network and so wanted a list of all ESXi hosts and their WWNs.  This struck me as another excellent scripting opportunity, so I pulled out my old Brocade 1:1 Zoning script and went to work.  I really only needed a one-liner for this situation, so I extracted my needed syntax from that script and quickly put it together.  Here's the (admittedly, ugly) command that I came up with:

get-vmhost | select name,@{N="WWNs";E={($_.ExtensionData.config.storagedevice.hostbusadapter.PortWorldWideName | % {("{0:X}"-f$_ -split '(..)' | ? {$_}) -join ':'}) -join ", "}}

That'll spit out a list with two columns, ESXi host names in one, and a comma separated list of WWNs in the other.  So, what's that ugly expression in that select statement doing?  Well, it's collecting all of the port WWNs on that ESXi host, then converting each one from decimal into hexadecimal (the "{0:X}"…

I was building a VDI solution for one of my customers that leveraged App Volumes to build RDSH servers, which in turn presented applications via Horizon (is it still fair to call it a VDI solution if there's no desktop OS involved?).  We were managing the user experience persistence via User Environment Manager, so the RDSH servers were stateless and no unique data would ever live on any of them.  It's a really cool solution, but we ran into that classic VDI issue: slow logons.

Fortunately, since that is such a classic issue, there's a huge list of things to do to alleviate it.  In this case, since the user's profile lives independently of the server to which they've logged on, we have a really powerful tool available to us: the Mandatory Profile.

Windows does a lot of profile customization when a user logs in for the first time, which is great on a persistent desktop!  In a nonpersistent environment (which, for all intents and purposes, any RDSH solution is, becau…

One of my customers has been deploying App Volumes, using their SQL Always On availability group for the database.  They pointed at the listener and everything looked good... until they initiated a failover.  Fortunately, the fix was really simple, although two part.

First, we noticed that the ODBC connections were using the default "SQL Server" driver.  That immediately stood out to us as a problem, as that driver can't handle AO failovers (exactly what we saw).  So, we switched it over to the Microsoft ODBC Driver 13 for SQL Server driver and figured that we'd be good to go.  Almost.

When we attempted to connect to the App Volumes Manager, we received a DB access error.  We saw that it was using the correct driver (yay!), but it was getting a credentials error because it was trying to use the server's computer account.  Some quick google searching pointed me at an article about changing database credentials in App Volumes, but that was for a fairly old version …

As a next-gen firewall, NSX allows us to get very dynamic with our firewall rules and create complex behaviors out of comparably very few rules.  Let's look at the same example 3 Tier Application rule set that I wrote about in my post about the Direction and Applied To fields in NSX Firewall rules post, and look at how these groups could be configured and then we can look at how these rules could be made even more secure.  First though, here's the example rule set:

SourceDestinationActionApplied ToAnyInfraServicesAllowDFWClient DevicesWebAllowWebDB, AppWebDenyWebWebAppAllowAppDB