Deleting Lots of Files from a VMFS Volume
One of my customers recently encountered an interesting problem. They encountered an issue that caused their Unidesk desktop backups to fail to delete from the archive datastore. As such, that datastore filled up and backups were unable to continue. The customer was working directly with Unidesk and VMware to resolve the issue, but in the meantime needed an easy way to purge all of the stale backups from their system.
A few months ago, I posted about setting up Persistent ESXi Scratch Space for hosts using SD cards for their installations. In that post, we covered how to mount a VMFS volume into a PowerCLI session. Once it's mounted, we created the per-host Scratch folders, but that same technique can be easily used here to help us find those stale backups. First, we ran this command to mount the "VDI-Archive1" datastore as a drive named DS:
Next, we had to understand how exactly we define a "stale backup". The "stale" part was easy to define - any file that was modified more than 5 days ago. The "backup" part wasn't too bad either; we just had to understand the folder structure that Unidesk uses for its desktop backups.
When Unidesk takes a backup of a desktop, we point that backup at a specific datastore (VDI-Archive1 in my example). On that datastore, Unidesk creates a folder structure like this:
[VDI-Archive1] Unidesk Backups/<CachePoint Name>/User/<VM Name>/UnideskBackupChain-<date>/ and stores its backup files in that folder.
Given that we were planning on running a command that would parse through a VMFS datastore and automatically delete a whole mess of files (we had hundreds of stale backup files to remove), we had to make sure that we didn't have any false positives. Everything that we're interested in deleting is under that UnideskBackupChain-<date> folder, but there's a lot of variability in the path (and even in that folder name). In order to ensure that we were only targeting files within the correct folder, we split the path and checked the 4th object in the resulting array, to make sure that it begins with UnideskBackupChain (after only listing objects within the "Unidesk Backups" folder of the target datastore). We then filter based on date, and have a list of all stale backup files. With the files identified, we simply pipe them into remove-item and are good to go.
I expect that someone reading this blog will be able to come up with something even better, given enough time. As always, any time you're playing around with commands (especially delete commands!), be very careful. This command worked for us in this particular situation, but be sure to test it thoroughly and be sure that you understand exactly what it's going to do before using it yourself!
A few months ago, I posted about setting up Persistent ESXi Scratch Space for hosts using SD cards for their installations. In that post, we covered how to mount a VMFS volume into a PowerCLI session. Once it's mounted, we created the per-host Scratch folders, but that same technique can be easily used here to help us find those stale backups. First, we ran this command to mount the "VDI-Archive1" datastore as a drive named DS:
New-PSDrive -Name "DS" -Root \ -PSProvider VimDatastore -Datastore (get-datastore VDI-Archive1)Next, we had to understand how exactly we define a "stale backup". The "stale" part was easy to define - any file that was modified more than 5 days ago. The "backup" part wasn't too bad either; we just had to understand the folder structure that Unidesk uses for its desktop backups.
When Unidesk takes a backup of a desktop, we point that backup at a specific datastore (VDI-Archive1 in my example). On that datastore, Unidesk creates a folder structure like this:
[VDI-Archive1] Unidesk Backups/<CachePoint Name>/User/<VM Name>/UnideskBackupChain-<date>/ and stores its backup files in that folder.
Given that we were planning on running a command that would parse through a VMFS datastore and automatically delete a whole mess of files (we had hundreds of stale backup files to remove), we had to make sure that we didn't have any false positives. Everything that we're interested in deleting is under that UnideskBackupChain-<date> folder, but there's a lot of variability in the path (and even in that folder name). In order to ensure that we were only targeting files within the correct folder, we split the path and checked the 4th object in the resulting array, to make sure that it begins with UnideskBackupChain (after only listing objects within the "Unidesk Backups" folder of the target datastore). We then filter based on date, and have a list of all stale backup files. With the files identified, we simply pipe them into remove-item and are good to go.
ls "DS:\Unidesk Backups" -recurse | ? {$_.FolderPath.split("/")[4] -like "UnideskBackupChain*"} | ? {$_.LastWriteTime -lt [DateTime]::Today.AddDays(-5)} | remove-item
ls (ls "DS:\Unidesk Backups\*\*\*\UnideskBackupChain*) | ? {$_.LastWriteTime -lt [DateTime]::Today.AddDays(-5)} | remove-itemI expect that someone reading this blog will be able to come up with something even better, given enough time. As always, any time you're playing around with commands (especially delete commands!), be very careful. This command worked for us in this particular situation, but be sure to test it thoroughly and be sure that you understand exactly what it's going to do before using it yourself!
Comments
Post a Comment
Sorry guys, I've been getting a lot of spam recently, so I've had to turn on comment moderation. I'll do my best to moderate them swiftly after they're submitted,