Migrating from one vCenter to Another, Improved
9/15/2016 Update: I've updated these scripts with some new functionality. I just posted a description of the new functionality as well as how to use the scripts in the context of a vCenter Upgrade.
A couple years ago, I made a blog post about a process that I use to migrate from one vCenter to another, and then I later created some scripts to facilitate that process. Those scripts worked fine for me, but others have found some weaknesses. Namely, they don't work if there are multiple folders in the inventory with the same name (such as VM\Project1\Production and VM\Project2\Production, where there are two folders named Production). Also, I never thought about VMs in vApps, which don't technically have a parent folder.
So, I've been working on updated scripts. I'm a little hesitant to post these, as I haven't actually used them to perform a migration yet, but there seems to be a lot of people who need them. I've put together two scripts again - a get script and a set script.
The Get script should be run on the source vCenter server and will generate a bunch of XML files that store many of that vCenter server's settings. It stores vCenter roles, Permission assignments, VM folder structure and VM inventory locations. This new version doesn't actually export the Folder or VM objects anymore (they've got way too much data for our purposes), instead it creates custom objects that store the VM's or Folder's name, as well as an array of strings to defines the exact inventory location of that object. So, an example VM would be defined by the following properties:
.Name: SacProj1Prod01
.FolderPath:@("Project1","Production")
The Get script takes two arguments: -directory and -includeTemplates. The -directory argument is the path to the output directory in which the script will generate its files. The -includeTemplates argument is actually a switch. If that switch is enabled, the script will convert all templates into VMs before taking the VM inventory. This is important because a Template is a vCenter based object, not an ESXi host object. This means that, in order to transition from one vCenter to another, you need a VM instead of a Template.
The Set script takes the xml files that the Get script generates and recreates those inventory objects (from a PowerCLI session that is connected to the destination vCenter server). I gave it a bunch of switches, as you're likely to run it twice during a normal migration. You'll probably run it once as a preparatory step, with the "-roles" "-permissions" and "-folders" arguments. Those three will recreate the VM folder structure from the source vCenter, recreate all of the roles from the source vCenter, and then reassign AD accounts those roles on the appropriate folders.
Once that run is completed, I highly recommend logging into the new vCenter and validating everything. Once you've verified that everything has been created successfully, you're ready to migrate some VMs into the new vCenter! I'd recommend using only test machines first, until you're sure that everything is working as intended, and make sure that you test it thoroughly. Once the inventory is populated with VMs (by migrating one or more hosts into it), you can move those VMs into the correct folders by using Set-SourceSettings.ps1 -VMs.
One thing to note - these scripts don't understand vApps. It turns out that vApps are super customizable with many variable options that a vendor can establish. This makes it really hard to anticipate what settings need to be recorded and then recreated... so for now, it doesn't. I've found some examples of PowerCLI scripts that can handle basic vApps, but I'm worried about missing settings that the script doesn't know about. So, for now, vApps must be recreated by hand.
Once again, these scripts are still undergoing development. I've only done very basic testing so far; please read through them and understand them before you try to use them, and as always test thoroughly. I think that they'll work, but that's no guarantee. If you find something that's broken, please feel free to either send me the fix or use GitHub to send me a pull request (if I've got my parlance correct).
Here's the Get-SourceSettings script:
And here's the Set-SourceSettings script:
A couple years ago, I made a blog post about a process that I use to migrate from one vCenter to another, and then I later created some scripts to facilitate that process. Those scripts worked fine for me, but others have found some weaknesses. Namely, they don't work if there are multiple folders in the inventory with the same name (such as VM\Project1\Production and VM\Project2\Production, where there are two folders named Production). Also, I never thought about VMs in vApps, which don't technically have a parent folder.
So, I've been working on updated scripts. I'm a little hesitant to post these, as I haven't actually used them to perform a migration yet, but there seems to be a lot of people who need them. I've put together two scripts again - a get script and a set script.
The Get script should be run on the source vCenter server and will generate a bunch of XML files that store many of that vCenter server's settings. It stores vCenter roles, Permission assignments, VM folder structure and VM inventory locations. This new version doesn't actually export the Folder or VM objects anymore (they've got way too much data for our purposes), instead it creates custom objects that store the VM's or Folder's name, as well as an array of strings to defines the exact inventory location of that object. So, an example VM would be defined by the following properties:
.Name: SacProj1Prod01
.FolderPath:@("Project1","Production")
The Get script takes two arguments: -directory and -includeTemplates. The -directory argument is the path to the output directory in which the script will generate its files. The -includeTemplates argument is actually a switch. If that switch is enabled, the script will convert all templates into VMs before taking the VM inventory. This is important because a Template is a vCenter based object, not an ESXi host object. This means that, in order to transition from one vCenter to another, you need a VM instead of a Template.
The Set script takes the xml files that the Get script generates and recreates those inventory objects (from a PowerCLI session that is connected to the destination vCenter server). I gave it a bunch of switches, as you're likely to run it twice during a normal migration. You'll probably run it once as a preparatory step, with the "-roles" "-permissions" and "-folders" arguments. Those three will recreate the VM folder structure from the source vCenter, recreate all of the roles from the source vCenter, and then reassign AD accounts those roles on the appropriate folders.
Once that run is completed, I highly recommend logging into the new vCenter and validating everything. Once you've verified that everything has been created successfully, you're ready to migrate some VMs into the new vCenter! I'd recommend using only test machines first, until you're sure that everything is working as intended, and make sure that you test it thoroughly. Once the inventory is populated with VMs (by migrating one or more hosts into it), you can move those VMs into the correct folders by using Set-SourceSettings.ps1 -VMs.
One thing to note - these scripts don't understand vApps. It turns out that vApps are super customizable with many variable options that a vendor can establish. This makes it really hard to anticipate what settings need to be recorded and then recreated... so for now, it doesn't. I've found some examples of PowerCLI scripts that can handle basic vApps, but I'm worried about missing settings that the script doesn't know about. So, for now, vApps must be recreated by hand.
Once again, these scripts are still undergoing development. I've only done very basic testing so far; please read through them and understand them before you try to use them, and as always test thoroughly. I think that they'll work, but that's no guarantee. If you find something that's broken, please feel free to either send me the fix or use GitHub to send me a pull request (if I've got my parlance correct).
Here's the Get-SourceSettings script:
And here's the Set-SourceSettings script:
The Get script errors out in the VM loop if you don't have any vApps.
ReplyDeleteOh, thanks for pointing that out! Man, these scripts have really highlighted the problems that come from developing a solution in a single environment. I've just added some logic that should solve that problem.
DeleteHi, I have been trying to use your scripts for the migration purpose, but I found som problems.
ReplyDeleteFirst off it still fails on the vApps, so i disabled that in the script.
Second the Set-SourceSettings -Folders, creates all folders with the following format:@{path=@{path=Star}}
Where Star is the Folder name :)
I havent investigated this heavily as to why yet, maybe you have som comments.
Excellent scripts by the way, all other functions seems ok so far, saves a lot of time !
Best Regards
Tor-Ivar
The Get script is still failing if there's no vApps? What message is it throwing?
DeleteI've just changed the set script. I think the problem was that the function was being passed an object with a .Path parameter (which was an array of folder names); it just expected the array of folder names. I've just changed line 39 so that it passes the function the array directly, instead of passing it the whole object.
Thanks for the feedback; I really appreciate it. I don't have a good situation to test these at the moment (as you can tell...) but I expect to be using them a lot over the coming year.
Jason,
ReplyDeleteFound a couple issues with the scripts you posted:
In Set-SourceSettings:
Change foreach ($thisPermission in $allPermissions)
to foreach ($thisPermission in $appPermissions)
Change (!($VMs -and $folders -and $permissions -and $roles)
to (!($VMs -and $folders -and $permissions -and $roles))
Ron
Thanks for the fixes!
DeleteI got some errors sin Set-SourceSettings along the lines "cannot find -principal....." in the set permissions part
ReplyDelete$target | new-vipermission -role $thisPermission.role -principal $thisPermission.principal.
when the principal in the xml input is in DOMAIN\username format.
And also in the set-sourceSettings isn't it better
$directory = $(read-host "Enter local INPUT directory")
??
Hah, yes about the $directory line. Copy-pasta strikes again!
DeleteRegarding your missing principals part, could it be something as simple as your new vCenter not being joined to the domain that those accounts reside in?
powercli 6.3
ReplyDeletemove-vm -folder "invalid argument" had to alter to move-vm -destination and it worked like a champ.
Thanks!
Jason,
ReplyDeleteThanks for the scripts. It saved me a lot of time. I had to tweak the rebuild permission section since vCenter permission and VM folder permission both uses permission.type Folder.
Here's my version of rebuild permission:
#Rebuild Permissions
$i = 0
foreach ($thisPermission in $allPermissions)
{
write-progress -Activity "Building permissions" -percentComplete ($i / $allPermissions.count * 100)
$target = ""
if ($thisPermission.entity -eq "Datacenters")
{
#permission is assigned to vCenter
$target = $thisPermission.entity
}
else
{
switch ($thisPermission.type)
{
#permission is assigned to Datacenter
Datacenter {$target = get-datacenter $dCenterNew; break}
#permission is assigned to Cluster
ClusterComputeResource {$target = get-cluster $clusterNew; break}
#permission is assigned to VMHost
HostSystem {$target = get-vmhost $thisPermission.entity; break}
#permission is assigned to a folder, use make-folder to get the precise folder
Folder {$target = make-Parentfolder -inFolderArray $thisPermission.entity; break}
#permission is assigned to VM
VirtualMachine {$target = get-vm $thisPermission.entity -Location $clusterNew ; break}
#error message
default {"Unexpected permission target, $($thisPermission.type)"}
}
}
if ($target)
{
new-vipermission -role $thisPermission.role -principal $thisPermission.principal -Entity $target -propagate $thisPermission.propagate
}
else
{
write-error "Unable to find permission object $($thisPermission.entity)"
}
$i++
}
My source and destination Datacenter and Cluster had different names, so I used variables.
How hard is it to make this script support multiple datacenters in the same vcenter?
ReplyDeleteThis comment has been removed by the author.
DeleteI would guess that you've got some permissions assigned on an object that I didn't expect when I wrote the script. If I were you, I'd add an output line to look at $thisFolder. That line is how it checks to see if we're at the "root" object of the datacenter and construct an absolute path to whatever folder it has been fed. It just keeps on getting the parent object until it gets down to the "VM" folder (if my memory serves), as determined by looking at the grandparent's .id parameter.
DeleteHi Jason, thanks for your quick answer. I deleted my previous entry previously to your post because i saw that the errors appear just in root folder. Anyway, to give context to your answer, i'll put it again. The error that i saw was:
ReplyDeleteThe property 'parent' cannot be found on this object. Verify that the property
exists.
At C:\Export_Source_vCenter_Config.ps1:38 char:24
+ while ($folderArray[-1].parent.parent.id -match "Folder")
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], PropertyNotFoundException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
And, as you said previously, reviewing the objects with fail we have:
ParentId :
Parent :
IsChildTypeVm : False
IsChildTypeComputeResource : False
IsChildTypeDatacenter : True
IsChildTypeDatastore : False
Type : Datacenter
Name : Datacenters
CustomFields : {}
ExtensionData : VMware.Vim.Folder
Id : Folder-group-d1
Uid : /VIServer=@:443/Folder=Folder-group-d1/
Client : VMware.VimAutomation.ViCore.Impl.V1.VimClient
As you can see all this items have the Parent property empty and from here the error.
Thanks again for your help and best regards.
Hello Jason. Just wanted to say thank you for your scripts. They really saved the day.
ReplyDeleteWe wanted to migrate from VCenter 5.1 to 6.0 but the problem was that we were using DB2 in 5.1. DB2 is not supported anymore and there is no way to make an in-place upgrade with the VCenter installer. The only way to go to 6.0 if you have 5.1 with DB2 is a new installation with whatever that brings (all folders, roles etc gone).
So, I am really glad that I found your blog post. The migration seems to be succesfull. =)
I have 3 comments though:
1) During import I was getting this error message. It didn't seem to affect the import whatsoever... but here it is:
get-datacenter : 15/12/2016 15:41:11 Get-Datacenter Server blah is not connected.
At C:\Set-SourceSettings.ps1:31 char:18
+ $parentFolder = get-datacenter $datacenter | get-folder $folderString
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Get-Datacenter], ViServerConnectionException
+ FullyQualifiedErrorId : Client20_ConnectivityServiceImpl_TryVerifyIsConnected_NotConnected,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetDatacenter
2) The templates switch didn't work. Not a biggie (I had only a few templates that I found later in the datastores and imported them manually). However, in the datastores I found the template files (.vmtx). That means there was no conversion to VMs done by your script. The templates xml file was exported in the backup directory though...
3) A recommendation based on my experience. Don't forget to backup the VDS configuration! I had completely forgotten to do it and I had to roll back the old snapshot, export it, go to the new one and import it. It would be cool if you could add a switch in your script to that too.
Thanks again and best regards =)
I'm glad to hear that you found the scripts helpful! I'm not sure why the template conversion didn't work - it may be a compatibility issues with 5.1, as I did all of my testing on 5.5.
DeleteIt looks like I messed up my variable scoping in the make-ParentFolder method of the script... I'll have to take a look at that and see if I can straighten it out.
Regarding the VDS, I wrote a separate blog post about the process that I used when using these scripts at https://virtuallyjason.blogspot.com/2016/09/migrating-from-vcenter-55-on-windows-to.html. It includes preparing the new vCenter with a functional VDS (I use this script https://virtuallyjason.blogspot.com/2015/10/script-to-make-distributed-switch-port.html) before performing the migration.
Jason, I want to thank you for your scripts you posted (on the VMware community forums, I believe. I wish I would have found this blog entry a few months ago. I used your script as a starting place, and modified them to work with our environment, which is completely dvSwitch based. I also ran into the issues with folders with the same name, and although it didn't impact my operations team that much, it did impact some of our business owners that we maintain the virtualization layer for.
ReplyDeleteAnyway, I believe the work I did to extend your scripts to copy dvSwitches, manage the uplinks to ensure there is no interruption to VM connectivity, and recreate DRS resource pools, would be of great use to others. However, I am a sloppy scritper, and they need to be clean up. Please let me know if you would like my changes, to provide back to the community.
VMware PowerCLI 11.0.0 build 10380590
ReplyDeleteHi Jason, First of all thanks for this script. It worked really well and saved a lot of time. I'd like to share my experience on "VM Placement". Line no 125 - Move-vm , you have to use both -Destination ( ESXi host ) and - Inventorylocation ( Foldername ) in order to perform successful migration. Other wise it won't work for vm placement. It should be "$foundVM | move-vm -destination $foundVM.VMHost -Inventorylocation $ParentFolder". Pls update this. Thanks once again.
Thanks for the tip, it looks like move-vm has changed over the years. I am curious about what your use case is for this script, given the improvements that VMware has made to the VCSA deployment's migration abilities.
Delete