Infrastructure Scalability and Datacenter Design for VDI

We’re all familiar with the concept of the core four resources: CPU, Memory, Network and Storage.  These days, we tend to combine CPU and Memory into a single “Compute” resource (since they are both contained within a single object, the ESXi host), but the concept still holds.  When we’re talking about resource utilization for a VDI solution, we’re usually concerned with Compute resources and Storage resources; PCoIP, RDP, ICA – they’re all pretty efficient protocols, so even a single gigabit adapter provides more than enough bandwidth for the VMs on most VDI ESXi hosts (IP based storage, on the other hand, might see more benefit from additional bandwidth).

So, how do we scale our Compute and Storage resources, as the project advances from POC to Pilot to a phased production rollout?  The answer to that question depends on how your infrastructure was designed – it was designed to be scalable, right?  At a high level, I’m seeing two different movements within the industry right now – call them discrete infrastructure and converged infrastructure.  Each one scales in different ways, and there are some very interesting things happening in both spaces right now.

Discrete infrastructure refers to those designs that use a collection of specialized, purpose built boxes.  This methodology was traditionally used in the virtualization space, with dedicated systems providing the Compute resources (the ESXi servers) and separate dedicated systems providing the Storage resources (the SAN).  This infrastructure includes using rack mounted or blade servers with a separate storage device.  This design philosophy traces its roots back to the Mainframe, where all of the various resource components are tied together into a single powerful datacenter which can be accessed remotely.

Converged infrastructure, on the other hand, brings all of the resources together into one system.  A single box has Compute and Storage resources (and, in some cases, Networking resources as well), and is generally both modular and self-contained.  This design philosophy is more in line with the Desktop/Smartphone model where we see self-contained computing resources distributed to the points where they are needed (which could very well be a single datacenter, although I see more movement towards a distributed model with converged infrastructure).

How do these two classes of infrastructure scale?  Well, it depends.  If designed well, both can be easily scaled, but each has its own challenges.

Discrete infrastructure gives us finer control over the individual resource components.  Are you running out of CPU/Memory?  Just buy another blade; assuming that your storage is fine, it’s a comparably cheap upgrade.  Need more storage performance?  Well, that can get tricky – and that’s the situation in which many people find themselves when expanding a VDI deployment.  Adding spindles to a big storage array can add some performance… but the bulk of your storage performance isn’t going to come from those spindles, it’s from the storage controllers and their cache (usually the Write Cache, to be more specific).  Depending on your SAN solution, adding another storage controller can be very expensive.  So this model scales very easily, up to the point where more storage performance is required.  At that point, a large investment is required in order to add another large chunk of storage performance.

Converged infrastructure, on the other hand, bundles all of those resources together into a single system.  If you need more CPU/Memory, you need to buy another server, which brings with it another chunk of storage.  This makes that server more expensive than one that only provides Compute resources, but it means that the Storage resources are now scaling linearly along with the Compute resources.  In a well understood VDI environment, where the Compute and Storage resource requirements are known, a good design on converged infrastructure can fully utilize each node.  This allows the environment to take advantage of that linear growth capability; the customer does not need to overprovision their storage in the initial VDI phases, as the storage grows along with the compute resources.  This efficiency requires careful planning and assessment though, as both the Compute and Storage requirements in the environment must be well understood.

Another interesting development that has come about recently is the hybridization of the two models.  Call it a software converged infrastructure.  In that model, the physical infrastructure can be discrete – with dedicated compute systems and dedicated storage systems.  By running virtual appliances though, each compute system can run its own optimized storage (sometimes in RAM for blazing fast speeds) creating a converged-like scenario.  This allows for a more linear growth in performance, as each Compute node also brings with it Storage performance, while utilizing a customer’s pre-existing investment in a large-scale SAN.

Any way about it, the infrastructure must be scalable for a successful VDI deployment.  Different designs will scale in very different fashions. Since VDI tends to scale much more aggressively than VSI, it is important to understand the scalability implications of any specific design and to plan for easy growth as more and more of a given enterprise moves to virtual desktops.