Virtually Jason

I've been migrating some large file servers from a customer's environment into a service provider's cloud.  Instead of just transferring the VMs (which would literally take weeks in some cases), we've provisioned new VMs and are replicating the data.  Then, during an outage window, we do a differential copy, shut down the old server and give the old server's name to the new server.  This also has the benefit of allowing us to get all of those older file servers up on Windows 2012 R2, so it's a win-win.  Well, mostly. One of these file servers has some very strict NTFS permissions due to the presence of sensitive data.  In this case, the only permissions on the folder were a Full Control for the Domain Admins and a Full Control for the group who owns the data (we'll call them the Finance Department).  We used good old Robocopy  (taking advantage of the new /MT switch to multithread it for added performance!) to copy the data and all permissions from t...

One of my customers has to an application with strict concurrency limits in its licensing.  Historically, they've worked around those concurrency limits by having a limited number of workstations from which to launch that application.  When a user needed it, they would just go to one of those workstations and do their work.  This created a serious pain point though, as users would frequently lock the workstation rather than log out of it, leaving their coworkers with one fewer workstation at that location (there are far more users then computers at these locations). VDI allows us to very easily work around that later problem, as each user can now get their own desktop.  The location is still limited by the number of terminals that it has, however a user can now simply restart the terminal in order to connect to their own desktop without causing their coworker to lose any work.  This flexibility has introduced the pain from this particular applications concurre...

One of my customers recently had a major issue; due to a fault in their cooling system, their server room overheated and many devices turned themselves off in order to avoid damage... including one of their SANs.  After resolving the cooling issue, the environment largely came back up without serious incident, however there was one lingering problem. Several VMs in the environment could not connect to their distributed virtual switch.  Each VM was associated with its port group correctly, however the "connected" checkbox was cleared.  When we tried enabling that option, the task failed with an error that read: Invalid Configuration for Device '0' Some quick googling revealed a VMware KB Article about this particular issue, with several different workarounds suggested.  They ranged from easy (move the VM to a different Port Group and back) to arduous and time consuming (remove the NIC from the VM, then add a new NIC and reassign its network identity to the new NI...

One of my customers offers a hosting solution.  One of their larger customers came to them with a request recently - they wanted a 5 second boot delay on all of their VMs.  This customer had previously run all of their VMs in their own vCenter and needed a way to get access to the VMs' BIOS settings now that they had the vCloud Director subset of actions.  A boot delay was deemed the best solution.  This presented another excellent scripting opportunity - a very simple task that needed to be repeated hundreds of times. My customer was understandably hesitant to let a script just run rampant through their vCenter, and so I put together the following.  It only targets VMs in the named Resource Pool and can be further filtered by passing it a VM's name (or part of a name) with the -VM parameter.  After that, it loops through the list of all returned VMs and checks their boot delay setting.  If the VM doesn't have the correct Boot Delay, the script promp...

One of my customers recently asked me a question about his VDI environment.  I had given him a copy of my View 5 network flows diagram , but he was curious about what those flows actually do during day to day operations.  This struck me as a pretty important question, as the diagram is useful by itself when planning a new deployment, but when troubleshooting a problem it doesn't really help you to know where to look.  So, I typed up some explanations of what the system is doing on the back end during some common front end operations.  Since that's useful info, I figured that I should probably post it here in case it can help anyone else. External User Connections The External View Client initiates the TCP 443 connection to the Security Server.  This connection is used until the user selects the desktop to which they want to connect After the user initiates the TCP 443 connection, the Security Server connects to the Connection Server The Connection S...

vCloud Director is a great tool, but I feel like some people aren’t always using it in the precise way that it was designed.  When you want to perform maintenance on a VM that vCD manages, you’re probably going to have to have that VM powered down.  So, what do you do?  You open your vApp, right click on your VM and select Power Off, right?  Well, I’ve seen a lot of people do just that, but it’s not really what I’d recommend.  Remember, Power Off is not Shutdown; it’s just going to perform a hard power off and your VM might have some issues after that. So, what do you do?  There’s no “Shutdown” option on that Right Click menu in vCD.  Since you want to keep your servers in tip-top shape, you open the VM console and then issue a command inside the guest OS (good ole “shutdown /s” anyone?).  The VM goes down gracefully and you can now manage it… but the vApp might be in an interesting state: “partially running” which can prevent you from changi...

Sometimes, we write scripts that we want to schedule.  For example, I recently put together a small script that checks an environment for Snapshots that are older than 7 days and then emails the sys admins a report if it finds any.  Alan Renouf put together a quick guide on how to run PowerShell scripts with Windows Task Scheduler back in 2009, but I ran into a hiccup while trying to use that procedure the other day. Alan suggests two possible methods for accomplishing this task.  One involves editing your scripts to add the required PS Snapins and the other involves launching the PowerShell session from a pre-configured Console File.  I decided that I’d had enough of messing around with Add-PSSnapin, so I decided to experiment with the PowerShell Console option.  The example syntax that he provides is basically: Powershell.exe –PSConsoleFile myConsole.psc1 & myScript.ps1 Which didn’t exactly work for me.  When I executed this at a command li...

How often do we, as system admins, find ourselves with a strange error that might be due to network connectivity?  Pretty often, in my experience.  While Ping is a great tool for validating basic routing, we live in a world of firewalls – be they physically on the network, virtually on the network, or software installed on the server.  When diagnosing network connectivity issues, Ping is not good enough, as that only tests the ICMP network flow.  You can’t ping a specific port in Windows, so if you’re looking to test something like HTTPS (TCP 443), with Ping you’re out of luck.  Fortunately, there are many other tools available. My favorite tool is PortQry  (that link will work as long as Microsoft graces us with a consistent URL… so until tomorrow, probably).  PortQry was originally a SysInternals tool (developed for Windows 2000!) and is still one of the best.  You’ll have to download it, but it’s a simple executable that you can invoke fro...

One of my customers was upgrading their Unidesk environment to 2.6 and we ran into a problem.  It proved to be an incredibly specific problem and not at all tied to Unidesk (just tied to the way they push this particular upgrade), but the troubleshooting process was very interesting and so I think it's worth putting this knowledge out there. As part of the upgrade, the Unidesk Management Appliance needs to install an OVF of itself (which is an uncommon but not unprecedented behavior for virtual appliances).  That is the step that was failing.  The most common source of that failure is a firewall; if port 443 is blocked between the Management Appliance and the ESXi host, that deployment will fail.  We went round and round with the network team, checking firewalls and couldn’t find any records of ports being blocked, but we couldn’t communicate with the ESXi hosts (although we could communicate with the vCenter server just fine). Eventually, we did a TCPDump ...

One of my customers was recently troubleshooting a network issue and had tracked it down to a single VMware MAC Address.  They tasked me with finding that VM that had that MAC address assigned, which is not a simple process through the GUI.  Fortunately, it's a very easy problem to solve with a script, and so PowerCLI to the rescue. All that I did with this script was get all of the VMs from vCenter, then loop through each one getting all of their NICs.  I then compared the MAC of that NIC against the MAC that the customer had identified, and we very swiftly had identified the problem VM. $MacOfInterest = "00:50:56:7b:00:00" $allVMs = Get-VM foreach ($thisVM in $allVMs) { foreach ($thisAdapter in ($thisVM | Get-NetworkAdapter)) { if ($thisAdapter.MacAddress -eq $MacOfInterest) { echo $thisVM.name } } }

One of the big new features of View 6 is the ability to use Terminal Server… err… I mean Remote Desktop Session Host RemoteApp applications .  As you’ve probably seen, the application presentation is pretty slick, behaving very much like natively installed applications even though they’re running on a server in the datacenter.  This allows for neat little tricks like launching Windows applications on an Android device.  In the demonstrations, this feature is just about always paired with Workspace 2.0, showing those RDSH applications in the Workspace 2.0 launchpad. It isn’t really clear from the marketing that’s going on, but RDSH application support is a View 6 feature, not a Workspace 2 feature.  When you want to present a RemoteApp, you must register the RDSH server in View and create an Application Pool.  From there, you can optionally tie it into Workspace or you can just launch it directly from the View Client – your choice.  So, how do you insta...

One of my customers uses the Sophos UTM virtual appliance as a firewall/router.  It was originally set up as an Astaro version 8 virtual appliance and has been upgraded in place since then.  When that appliance was first made available, it used the Flexible network adapter by default, although the recommendation now is to use vmxnet3 for better performance.  Unfortunately, during the upgrade process, this change never went into place. We’re trying to help this customer to resolve some performance problems and have identified that network adapter type as a potential bottleneck in their environment.  Before changing such a vital piece of a live environment, I decided to prototype it in the lab.  It turns out that Sophos no longer provides the virtual appliance , instead providing an ISO to install the software onto a VMware virtual machine.  No problem, I thought, and so I created the VM in my lab, aiming to match the appliance’s configuration settings. ...

When planning a VDI deployment, we all know that it’s important to define your use cases.  VMware’s methodology defines three main categories for these use cases: Task Worker, Knowledge Worker and Power User.  I see a lot of confusion in the field though – these are not use cases themselves.  I have yet to find an organization that can neatly divide their workforce into three use cases.  Instead, those categories are really just shorthand to allow us to roughly describe the resources that are required by the use case. Most environments will have several Task Worker use cases.  There might be a Call Center Task Worker use case, a Data Entry Clerk Task Worker use case and a Secretary Task Worker use case, all within the same organization.  This is perfectly normal, as “Task Worker” really means that the use case employs a very limited set of applications and does not require many resources.  So, how do you actually define a use case then? A us...

When working with VMware View (or any VDI solution for that matter), you’ll eventually have to deal with certificates.  Hopefully your organization has a skilled web or security team who can help you install the certificate onto the Connection Servers and Security Servers… but I find that that’s not usually the case.  Usually, someone either just downloads the certificate from GoDaddy (or whatever authority they use) or gives me the credentials so that I can download it myself. The problem is, what you download from GoDaddy isn’t going to work, at least not by itself.  There basically two parts of a certificate: the public key and the private key.  When you go to GoDaddy and launch their interface and download your certificate, you’re downloading the public key (and any intermediate certificates that are required to establish a chain of trust).  In order to install and use that certificate, you’ll need to provide the corresponding private key (which was used i...

I was working with one of my ProfileUnity customers and they decided that they wanted to change the file server for their user profile repository.  This path gets used in a lot of different places of a configuration.  The easiest way to change it in the GUI is to just create a new configuration (using the wizard) and specify the new file server.  In this case, that wasn’t really an option as they had a very developed configuration with a lot of customized settings that we didn’t want to recreate.  Fortunately, there’s a nice easy work around to make such changes. From the main ProfileUnity page, just download your configuration.  Instead of saving it as an .ini (like you would when transferring it into your domain\netlogin\ProfileUnity folder, export it as a .json file.  This .json file is ultimately just a text file… which means that you can open it up in your text editor of choice and use “find and replace” to great effect. When doing this, bear in mi...

I recently had the chance to do some testing with PernixData ’s vSphere solution and it was very educational.  It worked great, doing exactly what it was supposed to do… and it also got me thinking about the nature of storage caching. For just about ever, caching has been the technique that allows storage devices to provide the blazing performance that we all demand.  Monolithic SANs always have some amount of cache on their controllers.  What exactly does that cache do?  As the name implies, it caches data.  On those SANs, we would typically assign some amount of read cache and some amount of write cache (I’ve typically biased my vSphere storage devices heavily towards write cache, but opinions vary).  When a write request comes in, it is very quickly written to that cache and the acknowledgement is sent to the device that is performing the write.  The SAN’s job is then to destage the data from the cache onto the disks for long term storage, whic...

I was deploying Liquidware Labs ProfileUnity for a customer last week and we came across an interesting bug that I wanted to make a quick note of.  It's a cool tool for managing the general user experience; pretty much everything that lives in the user's profile.  It largely obviates the need for using the Group Policy editor and wading through the pages and pages of GPO settings that can apply to desktops by putting many of the most common settings into a much easier interface. One of the things that you can configure with ProfileUnity is shortcuts.  They can go on the Desktop or the Start Menu or any of the other places that you might want a shortcut to sit, and you can point them at pretty much anything that you'd like a shortcut to point at.  Great!  It's much easier to manage through their interface, as they allow you to filter who gets what shortcut and can automatically create them based on AD Group membership, etc.  It's great!  We did run in...

If you've looked at the scripts that I've written, you may have noticed that I use a fairly simple bit of syntax whenever I need to access a single property from all objects in an array (in PowerShell).  I just wrote that sentence and even I'm not entirely sure what it means, so I'd better use an example.  Try this: $events = get-eventlog -newest 10 $events.source What output did you see?  If you saw a list of 10 sources, congratulations, your PowerShell is at least version 3!  If you saw no output, then you should really update your PowerShell.  This is the functionality that I mean - the ability to easily generate an array of a single Property (in this case, Source) from an array of objects. I bring this up because that functionality is super useful.  For example, I was recently writing a script for a customer that involved analyzing a bunch of AD Users and I needed to manipulate a single Property and then select users based on that Property (I'm ...

I don’t know if anyone else was confused about this, but VMware’s View Client versioning scheme is a little counterintuitive (at least, for Windows clients).  Thus, if you go to download the View client , make sure that you download what you really want. Here’s the details: First, you'll see that there are two major versions: Version 1.0 and Version 2.0. Version 1.0 includes Windows View Client versions 5.2.1 and earlier, which corresponded nicely with Horizon View Server 5.2 and earlier.  After that, it gets a bit odd. Version 2.0 includes Windows View Client versions 5.3.0 and newer.  Sounds easy… but what’s newer than 5.3.0?  Well, 5.4.0 is newer.  But, so is 2.2.0… and 2.3.0. The reason that this happened is that some really cool features (mostly around media redirection and processing) have been developed as Client technologies without requiring changes on the server side (thus incrementing the Client version without incrementing the Server version)....

PowerShell's Sort-Object cmdlet is super useful, especially when preparing output for human consumption.  A few people have found my blog while looking for more information about its use, specifically while looking for how to sort by multiple columns (well, properties, technically).  I've never done so (much less written about it), so I hope those folks found answers elsewhere.  But, it got me curious... and it turns out that it's really easy. The Technet article on Sort-Object has the answer directly spelled out: just use commas to create a list of properties to sort by (in order to precedence).  Let's look at some examples!  First, prepare a variable with some good sortable data: $myData = get-eventlog System -newest 25 And then we can get to sorting!  Say you want to sort primarily by EntryType (Warning, Information, Error) and then by Source.  Easy-peasy: $myData | sort EntryType,Source How about if you want it to be in descending or...

Memory overcommitment remains a contentious topic, despite some really great studies on the topic.  Regarding VDI, I’ve heard opinions ranging from “0 memory overcommitment!” to “200%, 300%, it’s fine!”  I figured that I’d share my thoughts on the topic and see if anyone else wants to weigh in on the discussion.  First though, some points about the decision. The main argument that I’ve heard against memory overcommitment boils down to protecting the user experience.  Since virtual desktops have users actively logged into and using them, even slight performance degradation is immediately noticed.  We all know that hypervisor swapping is Very Bad for performance.  If you don’t overcommit your hosts’ memory, you won’t have swapping, so why overcommit? The simple answer is that, while 2 GB of RAM for a single desktop is no big deal, 2 GB of dedicated RAM for 1000 desktops is a lot harder to accept.  Memory overcommitment can be an important technique for...

One of my customers asked me to help them migrate their VMs from an ESX 4 environment into a brand new ESX 5 environment.  This new environment had brand new servers and storage (as well as fibre switches); they needed a forklift.  We discussed migration strategies (presenting some shared storage to both environments, etc.) but, due to a long list of very environment specific considerations, we determined that cold migrations during outage windows would be the best solution. As such, there was a fair amount of hands on work that would be required.  Given that a large amount of the work would need to be done after hours, automation became a high priority.  To that end, I put together a PowerCLI script that the migration team could use to perform these migrations.  This script is designed to be used by an administrator who can babysit the process, for it spits out its status updates and end results so that the administrator can validate the important settings. ...