Virtually Jason

One of my customers had an issue where a bunch of VMs were not able to vMotion, despite the hosts being configured correctly in all regards (other VMs using the same VDS Port Groups, for example, could vMotion onto and off of the host where these VMs were running).  When DRS (or an administrator) attempted a vMotion, a generic "A general system error occurred: vim.fault.NotFound" error message would be displayed. When I took a look at these VMs, I noticed something interesting (besides the fact that they were all on the same host); their VDS Port numbers were universally high, like in the 5000s.  This was particularly interesting because when I looked at the VDS itself, the highest numbered port on it was 4378.  I supposed that these ephemeral ports had somehow been assigned invalid port numbers, which was causing vMotion to fail when the new destination was unable to reserve that invalid number on the VDS.  Interestingly, all of these VMs were communicating just...

One of my customers recently experienced a strange issue.  One of their ESXi hosts had entered a problem state where Storage vMotion and vMotion were failing for all VMs on the host (vMotion was failing at 13%, which is an interesting spot).  We initially noticed the issue when Storage vMotion repeatedly threw an error for one of their VMs: A general system error occurred: Failed to create journal file provider: Failed to open "/var/log/vmware/journal/..." for write: There is no space left on the device. Well, that error seemed self explanatory.  I connected to the host's CLI (SSH was already enabled, which made that easier) and did a "vdf -h" to look at its file system.  I was surprised to find that none of the partitions were full, so I dug deeper. I decided to take a look at the vmkwarning log file, which is frequently a gold mine when troubleshooting ESXi host issues.  So, I did a quick "tail /var/log/vmkwarning.log" and, lo and  behold, we h...