Copying VM Folders and Permissions from One vCenter to Another
1/26/2016 Update: We've learned that this script doesn't elegantly handle the situation where multiple VM folders have the same name (but different parent folders). I've put together a basic script that should help recover from that (in the comments), but I haven't had a chance to thoroughly test it. That script is built to move all VMs from the "Discovered Virtual Machine" Folder back into their correct locations. I'm working on an updated pair of scripts that will handle this whole situation better.
2/9/2016 Update: I've published an updated version of the vCenter Migration Scripts - check it out instead of these!
One of my customers wanted to export their VM Folder structure and the associated permissions from one vCenter to another, in preparation for a vCenter forklift. Once the destination vCenter was fully prepared, the goal would be to migrate ESXi hosts with VMs into the new vCenter, then move the VMs into the appropriate folders. This situation is pretty much the epitome of why I like scripting; it's a long, arduous process that has tons of opportunity for human error to make subtle problems. So, I put together a pair of scripts.
The first script basically reads all of the data from the source vCenter server. It outputs a series of XML files that contain the VM folder structure, all of the vCenter Roles, the vCenter Permissions structure and what VMs are in what folders. As part of this migration process, templates will be converted back into VMs (so that they can easily traverse from one vCenter to the other), so I'm not worrying about using a separate command to capture Template locations as well.
The second script reads in all of those files and recreates all of the vCenter objects. It begins by creating the VM Folder structure, then creates new versions of all of the Roles from the previous vCenter and finally assigns those Roles to the appropriate Principals on the appropriate vCenter objects.
In order to run these scripts, we first converted the Templates into VMs by using this command:
get-template | set-template -tovm
Then, while connected to the Source vCenter, we ran the capture script:
get-sourcesettings.ps1 -d C:\temp -dc Sacramento
Next, while connected to the Destination vCenter, we ran the configuration script:
set-SourceSettings.ps1 -d C:\temp - dc Sacramento
At that point, the new vCenter was configured and so we began shuffling the ESXi hosts into it. Once all of the hosts (and their VMs) were in the environment, we placed the VMs back into their folders with this command:
foreach ($thisVM in (import-clixml C:\Temp\vm-locations.xml)) {get-vm $thisVM.name | move-vm -destination (get-folder $thisVM.folder)}
Then, we just converted all of the template VMs back into templates (they were easy to find, as they were in the "Templates" folder) and were good to go.
These scripts are formatted in a less-than-human-friendly manner in order to make it easy to pick them apart to manually run any one bit of it. Are you just interested in grabbing the roles? Well, grab the command line for the roles, as it's completely self contained. Same thing for the Folders or Permissions lines. This means that the line breaks due to the blog's width are particularly troublesome here, so be extra careful to remove any extraneous line breaks.
As always, these scripts worked for me in my situation, but that's no guarantee that they'll work for you, so test thoroughly and share any fixes that you make!
# Get-SourceSettings.ps1
# Exports source Folders/Roles/Permissions/VM Locations
param
(
[alias("d")]
$directory = $(read-host "Enter output directory"),
[alias("dc")]
$datacenter = $(read-host "Enter datacenter name")
)
$directory = $directory.trim("\")
new-item $directory -type directory -erroraction silentlycontinue
get-virole | where {$_.issystem -eq $false} | export-clixml $directory\roles.xml
Get-VIPermission | export-clixml $directory\permissions.xml
$dc = get-datacenter $datacenter
$dc | get-folder | where {$_.type -eq "VM"} | select name,parent | export-clixml -depth 5 $directory\folders.xml
$dc | get-vm | select name,folder | export-clixml -depth 6 $directory\vm-locations.xml
# Set-SourceSettings.ps1
# Run on destination vCenter to recreate VM folder structure and permissions
param
(
[alias("d")]
$directory = $(read-host "Enter input directory"),
[alias("dc")]
$datacenter = $(read-host "Enter datacenter name")
)
$directory = $directory.trim("\")
#Read in the folder structure from the Get-SourceSettings.ps1 script and create those VM Folders
foreach ($thisFolder in (import-clixml $directory\folders.xml | where {!($_.name -eq "vm")})) {(get-datacenter $datacenter) | get-folder $thisFolder.Parent | new-folder $thisFolder.Name -confirm:$false}
#Read in Roles from Get-sourceSettings.ps1 script and create those Roles
foreach ($thisRole in (import-clixml $directory\roles.xml)){if (!(get-virole $thisRole.name -erroraction silentlycontinue)){new-virole -name $thisRole.name -Privilege (get-viprivilege -id $thisRole.PrivilegeList)}}
#Read in Permissions from Get-sourceSettings.ps1 script and assign those permissions
foreach ($thisPerm in (import-clixml $directory\permissions.xml)) {get-folder $thisPerm.entity.name | new-vipermission -role $thisPerm.role -Principal $thisPerm.principal -propagate $thisPerm.Propagate}
#Read in the VM Folder locations and move the VMs to their Folders; only execute this line after all VMs have been moved into the new inventory
#$allVMs = import-clixml C:\Temp\vm-locations.xml
#foreach ($thisVM in $allVMs) {get-vm $thisVM.name | move-vm -destination (get-folder $thisVM.folder)}
2/9/2016 Update: I've published an updated version of the vCenter Migration Scripts - check it out instead of these!
One of my customers wanted to export their VM Folder structure and the associated permissions from one vCenter to another, in preparation for a vCenter forklift. Once the destination vCenter was fully prepared, the goal would be to migrate ESXi hosts with VMs into the new vCenter, then move the VMs into the appropriate folders. This situation is pretty much the epitome of why I like scripting; it's a long, arduous process that has tons of opportunity for human error to make subtle problems. So, I put together a pair of scripts.
The first script basically reads all of the data from the source vCenter server. It outputs a series of XML files that contain the VM folder structure, all of the vCenter Roles, the vCenter Permissions structure and what VMs are in what folders. As part of this migration process, templates will be converted back into VMs (so that they can easily traverse from one vCenter to the other), so I'm not worrying about using a separate command to capture Template locations as well.
The second script reads in all of those files and recreates all of the vCenter objects. It begins by creating the VM Folder structure, then creates new versions of all of the Roles from the previous vCenter and finally assigns those Roles to the appropriate Principals on the appropriate vCenter objects.
In order to run these scripts, we first converted the Templates into VMs by using this command:
get-template | set-template -tovm
Then, while connected to the Source vCenter, we ran the capture script:
get-sourcesettings.ps1 -d C:\temp -dc Sacramento
Next, while connected to the Destination vCenter, we ran the configuration script:
set-SourceSettings.ps1 -d C:\temp - dc Sacramento
At that point, the new vCenter was configured and so we began shuffling the ESXi hosts into it. Once all of the hosts (and their VMs) were in the environment, we placed the VMs back into their folders with this command:
foreach ($thisVM in (import-clixml C:\Temp\vm-locations.xml)) {get-vm $thisVM.name | move-vm -destination (get-folder $thisVM.folder)}
Then, we just converted all of the template VMs back into templates (they were easy to find, as they were in the "Templates" folder) and were good to go.
These scripts are formatted in a less-than-human-friendly manner in order to make it easy to pick them apart to manually run any one bit of it. Are you just interested in grabbing the roles? Well, grab the command line for the roles, as it's completely self contained. Same thing for the Folders or Permissions lines. This means that the line breaks due to the blog's width are particularly troublesome here, so be extra careful to remove any extraneous line breaks.
As always, these scripts worked for me in my situation, but that's no guarantee that they'll work for you, so test thoroughly and share any fixes that you make!
# Get-SourceSettings.ps1
# Exports source Folders/Roles/Permissions/VM Locations
param
(
[alias("d")]
$directory = $(read-host "Enter output directory"),
[alias("dc")]
$datacenter = $(read-host "Enter datacenter name")
)
$directory = $directory.trim("\")
new-item $directory -type directory -erroraction silentlycontinue
get-virole | where {$_.issystem -eq $false} | export-clixml $directory\roles.xml
Get-VIPermission | export-clixml $directory\permissions.xml
$dc = get-datacenter $datacenter
$dc | get-folder | where {$_.type -eq "VM"} | select name,parent | export-clixml -depth 5 $directory\folders.xml
$dc | get-vm | select name,folder | export-clixml -depth 6 $directory\vm-locations.xml
# Set-SourceSettings.ps1
# Run on destination vCenter to recreate VM folder structure and permissions
param
(
[alias("d")]
$directory = $(read-host "Enter input directory"),
[alias("dc")]
$datacenter = $(read-host "Enter datacenter name")
)
$directory = $directory.trim("\")
#Read in the folder structure from the Get-SourceSettings.ps1 script and create those VM Folders
foreach ($thisFolder in (import-clixml $directory\folders.xml | where {!($_.name -eq "vm")})) {(get-datacenter $datacenter) | get-folder $thisFolder.Parent | new-folder $thisFolder.Name -confirm:$false}
#Read in Roles from Get-sourceSettings.ps1 script and create those Roles
foreach ($thisRole in (import-clixml $directory\roles.xml)){if (!(get-virole $thisRole.name -erroraction silentlycontinue)){new-virole -name $thisRole.name -Privilege (get-viprivilege -id $thisRole.PrivilegeList)}}
#Read in Permissions from Get-sourceSettings.ps1 script and assign those permissions
foreach ($thisPerm in (import-clixml $directory\permissions.xml)) {get-folder $thisPerm.entity.name | new-vipermission -role $thisPerm.role -Principal $thisPerm.principal -propagate $thisPerm.Propagate}
#Read in the VM Folder locations and move the VMs to their Folders; only execute this line after all VMs have been moved into the new inventory
#$allVMs = import-clixml C:\Temp\vm-locations.xml
#foreach ($thisVM in $allVMs) {get-vm $thisVM.name | move-vm -destination (get-folder $thisVM.folder)}
is this done through powercLI or powerwhell? It has been so long since i scripted i forgot.
ReplyDeleteThese scripts use VMware's PowerCLI, which is a bunch of additional modules for PowerShell. I generally assume that you have a session open and that you've already used "connect-viserver" to connect to the appropriate vCenter before launching any of these scripts.
DeleteWe got it!
ReplyDeleteWe just had to change it to this for the variables-
for $directory we changed the syntax to $directory = "c:\temp" and this included the quotes. Same thing went for $datacenter, where it was $datacenter = "Pittsburg"
$directory = read-host "Enter output directory",
[alias("dc")]
$datacenter = read-host "Enter datacenter name"
Ah, I think that I see what was going on. When I execute these, I almost always specify my parameters right in the command or even just hard code a default value that's appropriate to the site where I develop the script. When I scrubbed this for posting on the web, I think that I failed to encapsulate the "read-host" cmdlets properly for the param block. It sounds like you worked around it, but I think that I fixed it in the actual post now.
DeleteWithin our vcenter, we have multiple datacenters. 28 to be exact. Is there a way to write the sript to include multiple datacenters?
ReplyDeleteThis could be modified to handle multiple datacenters, but I think it might be easier to just programatically call the script. I don't have a PowerCLI session in front of me to test this, but you could do something like this:
Deleteget-datacenter | % {Get-SourceSettings.ps1 -d "C:\Output\$($_.name)" -dc "$($_.name)"}
That should get all of the datacenters in your vCenter, then call the script for each one, making a new directory per datacenter under C:\Output. I'm not in the office today and so can't test that syntax, but I believe that it will work...
I got following errors when I execute the command
ReplyDeleteforeach ($thisVM in (import-clixml C:\Temp\vm-locations.xml)) {get-vm $thisVM.name | move-vm -destination (get-folder $thisVM.folder)}
"
Move-VM : Cannot convert 'System.Object[]' to the type
'VMware.VimAutomation.ViCore.Types.V1.Inventory.VIContainer' required by
parameter 'Destination'. Specified method is not supported.
At line:1 char:102
+ ... m -destination (get-folder $thisVM.folder)}
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Move-VM], ParameterBinding
Exception
+ FullyQualifiedErrorId : CannotConvertArgument,VMware.VimAutomation.ViCor
e.Cmdlets.Commands.MoveVM
Move-VM : Cannot convert 'System.Object[]' to the type
'VMware.VimAutomation.ViCore.Types.V1.Inventory.VIContainer' required by
parameter 'Destination'. Specified method is not supported.
At line:1 char:102
+ ... m -destination (get-folder $thisVM.folder)}
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Move-VM], ParameterBinding
Exception
+ FullyQualifiedErrorId : CannotConvertArgument,VMware.VimAutomation.ViCor
e.Cmdlets.Commands.MoveVM
Move-VM : Cannot convert 'System.Object[]' to the type
'VMware.VimAutomation.ViCore.Types.V1.Inventory.VIContainer' required by
parameter 'Destination'. Specified method is not supported.
At line:1 char:102
+ ... m -destination (get-folder $thisVM.folder)}
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Move-VM], ParameterBinding
Exception
+ FullyQualifiedErrorId : CannotConvertArgument,VMware.VimAutomation.ViCor
e.Cmdlets.Commands.MoveVM
"
yep.. me too. I thought upgrading powercli to version 6 would help but no.
ReplyDeleteHmm... it's been a while since I used that, but it worked fine for me when I did. Try using this line and see what it spits out - ideally, it'll just be a bunch of folder objects:
Deleteforeach ($thisVM in (import-clixml C:\Temp\vm-locations.xml)) {get-folder $thisVM.folder}
Based on those errors, it looks like it's not returning the VM folders that I was expecting it to return and is instead getting a PowerShell object. I'm wondering if maybe it can't fine those folders and it's getting a bunch of nulls? If that's the case, this test line will return no results (or maybe a bunch of errors; I'm on vacation at the moment and so can't test how this will behave).
thanks Jason! It's actually listing all folders, as partly shown below. It seems the -destination is expecting a path in a different way. Thanks for replying anyway! Enjoy vacation
ReplyDeleteFSU VM
FSU VM
File Servers and Transfers Datastore
File Servers and Transfers VM
File Servers and Transfers VM
File Servers and Transfers Datastore
File Servers and Transfers VM
File Servers and Transfers VM
File Servers and Transfers Datastore
File Servers and Transfers VM
File Servers and Transfers VM
Is it possible that you have multiple folders with the same name? I don't know what your folder hierarchy looks like, but could you have 2 "FSU VM" folders, each under a different parent folder? I'm curious about that and the "File Servers and Transfers VM" folders, mainly because they seem to come in pairs in the output. When I wrote that, I assumed that each folder name would be unique and so I only recorded the parent object of the VM.
DeleteDo this command:
get-folder "File Servers and Transfers VM"
If it returns 2 folders (or more), then that would explain the error that you're seeing. We'd be passing an array of folder objects to the move-vm cmdlet's -destination parameter, rather than the object that the cmdlet needs.
Unfortunately, if that's the case, we'd need some human interaction in order to determine which of the folders is the correct one for that VM. Something like what I've written below would at least notify the user that they've got some work to do. With some more work, you could even set it to generate a menu so that the user can select the right folder and then have it place it in there:
foreach ($thisVM in (import-clixml C:\Temp\vm-locations.xml))
{
$thisFolder = get-folder $thisVM.folder
if ($thisFolder.count -gt 1)
{
echo "Someone needs to put $($thisVM.name) into the right $($thisVM.folder) folder"
}
else
{
get-vm $thisVM.name | move-vm -destination $thisFolder
}
}
Many thanks again Jason! You are right and apologies for not having mentioned that earlier. I didn't know that duplicated folders could cause that mess. We have loads of duplicated folder names. Your strings above are good and in fact I'll use the output to write to a file the list of VMs that need manual intervention. The rest of the VMs are being moving correctly indeed. Many tahnks again and happy new year!
ReplyDeleteso if I'm following the thread correctly (and its quite possible I'm not) if you have the same folder names but under different parent folders there is no way to automatically put the vm's back in their original folders. Is that correct? Is that because the xml output does not capture the full path of each of the folders that the vm's live in or something else?
ReplyDeleteI have a huge pile of vm's that are now in the discovered vm's folder and only the xml file as a reference to where they originally lived and I"m not sure that's enough for me to put them all back correctly.
Thanks
Actually, I think that the data exists in the xml file, as that file is just the output of this command:
Deleteget-vm | select name,folder
The folder that is listed there is an actual PowerCLI Folder object and it has a .parent attribute that can be used to figure out what its parent folder is. I put together the below code that should place those VMs in the correct folders... but this is really untested. That third line should limit this script to only executing on a single VM (the first one that it finds in the Discovered Virtual Machine folder). If it works, you can commend out that line to let it run on all VMs that it finds in that folder. Please be careful with this; I literally wrote it this morning and don't have the opportunity to even do the basic testing that I do before I normally post things. Please look the code over to ensure that what it's doing makes sense before you run it.
$allVMs = import-clixml C:\Temp\vm-locations.xml
$missedVMs = get-folder "discovered virtual machine" | get-vm
$missedVMs = $missedVMs[0]
foreach ($thisVM in $missedVMs)
{
#Creates an array of folders back to the root folder
$folderArray = @()
$folderArray += ($allVMs | ? {$_.name -eq $thisVM.name}).folder
while ($folderArray[-1].name -ne "vm")
{
$folderArray += $folderArray[-1].parent
}
#Gets the precise folder by getting its parents all the way back from the root
[array]::Reverse($folderArray)
$parentFolder = $NULL
foreach ($thisFolder in $folderArray)
{
$parentFolder = $parentFolder | get-folder $thisFolder
}
$thisVM | move-vm -destination $parentFolder
}
Thanks Jason. I'll give this a crack and see what happens.
DeleteIt appears to me that this line is hanging: $folderArray += $folderArray[-1].parent quite possibly the whole loop surrounding it.
DeleteI bet that I know what happened. Export-clixml only goes 2 generations deep (I've learnt this morning while writing newer versions of these scripts). So, the VM object has a .parent property, which is a folder object. That folder object has a .parent property which, in a PowerCLI session, will be some other container Object... but export-clixml only stores a string with that object's name. I've added -depth switches to the post to record more generations while I work on a better solution. For now, try this modified version, which should kick out of that loop if the last object in the array doesn't have a parent object.
Delete$allVMs = import-clixml C:\Temp\vm-locations.xml
$missedVMs = get-folder "discovered virtual machine" | get-vm
$missedVMs = $missedVMs[0]
foreach ($thisVM in $missedVMs)
{
#Creates an array of folders back to the root folder
$folderArray = @()
$folderArray += ($allVMs | ? {$_.name -eq $thisVM.name}).folder
while ($folderArray[-1].name -ne "vm" -and $folderArray[-1].parent)
{
$folderArray += $folderArray[-1].parent
}
#Gets the precise folder by getting its parents all the way back from the root
[array]::Reverse($folderArray)
$parentFolder = $NULL
foreach ($thisFolder in $folderArray)
{
if ($thisFolder.name){$parentFolder = $parentFolder | get-folder $thisFolder.name}
else{$parentFolder = $parentFolder | get-folder $thisFolder}
}
$thisVM | move-vm -destination $parentFolder
}
Jason - that worked. Thanks so much you rock! Saved me bacon.
DeleteGreat! I'm glad to hear that it worked; now I just need to complete a proper update to these scripts to handle this situation more elegantly...
Deleteactually it only partially worked. I think its still getting hung up where there are the duplicate folder names in the datacenter I'm working in. I'm getting this error on most of the vm's
ReplyDeleteMove-VM : Cannot convert 'System.Object[]' to the type 'VMware.VimAutomation.ViCore.Types.V1.Inventory.VIContainer' required by parameter
'Destination'. Specified method is not supported.
At line:24 char:32
+ $thisVM | move-vm -destination $parentFolder
+ ~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Move-VM], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgument,VMware.VimAutomation.ViCore.Cmdlets.Commands.MoveVM
That error probably means that it's still getting an array of folders back, instead of a single one. Is it possible that you have a folder nested in another folder of the same name? It might be possible to solve that issue by using the -NoRecursion switch on the get-folder cmdlets, but you'd have to initially populate $parentFolder accurately instead of just starting it with a $NULL for that to work. I've taken a stab at that below, but if you don't have too many VMs that it stumbles on, it might be worth just biting the bullet and moving them by hand.
Delete$allVMs = import-clixml C:\Temp\vm-locations.xml
$missedVMs = get-folder "discovered virtual machine" | get-vm
$missedVMs = $missedVMs[0]
foreach ($thisVM in $missedVMs)
{
#Creates an array of folders back to the root folder
$folderArray = @()
$folderArray += ($allVMs | ? {$_.name -eq $thisVM.name}).folder
while ($folderArray[-1].name -ne "vm" -and $folderArray[-1].parent)
{
$folderArray += $folderArray[-1].parent
}
#Gets the precise folder by getting its parents all the way back from the root
[array]::Reverse($folderArray)
$parentFolder = get-folder $thisFolder[0]
foreach ($thisFolder in $folderArray[1..$folderArray.count-1)
{
if ($thisFolder.name){$parentFolder = $parentFolder | get-folder $thisFolder.name -noRecursion}
else{$parentFolder = $parentFolder | get-folder $thisFolder -noRecursion}
}
$thisVM | move-vm -destination $parentFolder
}
This comment has been removed by the author.
ReplyDeleteSo I have a folder structure similar to this:
ReplyDeleteVirtual Datacenter
Folder1
--->FolderA
--->FolderB
--->FolderC...
FolderA
FolderB
FolderC
Where folder1 is a project name and the a,b,c folders are different environments for that particular project. Then there are other folders at the same level as Folder1 that have the same names as the A, B, and C folders under folder one.
I hope that makes sense.
Is there a way to account for that as well? I limited the get-folder to a particular virtual datacenter by using the -Location parameter but the move-vm only has a -destination parameter not a location.
I'm so close I can taste it.
Thansk for all your help.
move-vm's -destination parameter takes a Container object, so if you can successfully retrieve that object (by limiting get-folder to the specific datacenter, for example), you can pass it to move-vm and place the VM in that specific object.
Deletethe problem is that within the datacenter i still have the same folder names but under different parent folders. It also may be that its getting confused by the varying folder depth's. I'm not sure. I'm going to have to look at it on monday. I owe my kids a ski weekend.
Deletethanks for your help jason.
This scrip runs a long time, and when I ctrl-c to kill it it and look at the XML files they are like 5 GB... is this normal or is something wrong? Source is 500 VMs and 30 hosts, maybe 50 folders and I would not say an unreasonable amount of custom permissions.
ReplyDeleteHmm.. 5 GB sounds excessive. I haven't had a chance to tweak the "-depth" option on all of the export-clixml cmdlets, but that's almost certainly the culprit. If all of the folders in your environment are uniquely named, you can just remove that argument and it'll work fine. If you do have folder names that get reused (like Dan mentioned above), you should set the "-depth" value so that it's high enough to trace from your deepest VM back to the root folder.
DeleteI had no idea that so many people needed scripts like these. I'll redouble my efforts to put together something more robust that can deal with these issues.
DeleteThis comment has been removed by the author.
ReplyDeleteThere is a vmware fling that creates entire inventories to port settings from one vcenter to another called inventory snapshot. this includes cluster datacenters etc
ReplyDeletelink to fling https://labs.vmware.com/flings/inventorysnapshot
These two Scripts - GetSourcesettings and Setsource settings are the best for getting VMfolder details, roles, permissions, vms belonging to each folder and then setting or exporting these details to the new vCenter. I used these two scripts and it worked like magic. I did my new install of vSphere 6.0U2 and had to recreate all the VM folders, roles etc.. If it was'nt for these scripts I would have done a lot of manual work recreating the folder structure. Thanks Jason, these two scripts helped me a lot!!!
ReplyDeleteYou're welcome - I'm glad that they were helpful to you!
DeleteHi, thanks for great script!
ReplyDeleteGot some warnings about obsolete parameter when running folder-permission restore. Running on PowerCLI 6.x so they must have changed some. Anyone got a good fix for this or is it safe to ignore?
The warnings I get on each permission is the 2 following.
WARNING: Parameter 'Principal' is obsolete. This parameter no longer accepts multiple values.
WARNING: Parameter 'Entity' is obsolete. This parameter no longer accepts multiple values.
/Z