VCenter Server Difficulties After Oracle Crash
Does your VCenter run on an Oracle backend? Has that Oracle server ever gone down unexpectedly and, even after the DBA swears it’s back up (and your ODBC connection tests successfully), your VCenter process continues to refuse to start? You too? Here’s some details about the situation that I experienced and how we got around it. First, the identifying information:
If you look on your VCenter server in C:\ProgramData\VMware\VMware VirtualCenter\Logs, you will see a lot of log files. The VPXD-### log files are the ones that I looked at (the most recent one will be a .log file, the others are compressed tar.gz files that can be opened in Windows with 7-Zip, amongst other tools). If your VCenter service is refusing to start but your ODBC Test Connection is successful, take a look at those files. I found the following interesting errors in mine:
[Vdb::LockRepositoryHelper] SQLExecDirect failed… …Failed to Lock vCenter Repository!!...
Unable to get exclusive access to vCenter Repository. Please check if another vCenter instance is running against the same database schema.
Well, I knew for a fact that there was no other vCenter instance running against this database, so I dug in a bit deeper. These errors continued popping up (and the service continued failing) even after the DBA restarted the database instance (as the VMware KB article suggests). On a whim, I decided to reboot both the VCenter server and the Oracle server (this server is exclusively used by VCenter, so it wasn’t a big deal)… and it came back up successfully! Apparently, rebooting the database instance is not sufficient to clear up the stale sessions… rebooting the actual server, on the other hand, worked like a charm.
Hopefully you never need to do this.