Fixing the Failed Status of imageinfo on Exadata Compute Node

I was conducting regular OS patching of all the Exadata components when one of compute nodes caused imageinfo status to report a failure. Here are the steps I took to resolve the issue.

Initially, I thought the problem was caused by the lack of space in root “/” which interrupted the patching, but the node was functioning and GI and RDBMS stacks were patched without further issues.

 However, the services on the dbserver couldn’t be started and instead, failed with these messages:

DBMCLI> ALTER DBSERVER STARTUP SERVICES ALL

Starting the RS and MS services...
DBM-01512: Cannot start a new Restart Server (RS).  Exception received: DBM-02624: Error while executing command /opt/oracle/dbserver/dbms/deploy/scripts/unix/rs/startupRS.sh run_all -- timeout.

 
Validations failed on misceachboot step:

/opt/oracle.cellos/validations/bin/vldrun.pl -all
Logging started to /var/log/cellos/validations.log
Command line is /opt/oracle.cellos/validations/bin/vldrun.pl -all
Run validation ipmisettings - PASSED
Run validation misceachboot - FAILED
Check log in /var/log/cellos/validations/misceachboot.log
Do you want to interrupt all validations (y/n) [y]:

There are several MOS notes about similar issues and I tried an ILOM reset and a server reboot,
but they didn’t help start the services. Nor did they resolve the issue with a validation check.

 
Finally, with help of “How To – Re-Install Exadata Management Package (dbmmgmt rpm) On Exadata Compute Nodes (Doc ID 2134218.1)”, I reinstalled the package “exadata-dbmmgmt”which successfully started RS and MS services:

# rpm -ivh /tmp/exadata-dbmmgmt-21.2.5.0.0.211013-1.noarch.rpm
Preparing...                          ################################# [100%]
2021-11-14 18:52:31 -0600: Pre Installation steps in progress ...
2021-11-14 18:52:32 -0600: This is a fresh install.
Updating / installing...
   1:exadata-dbmmgmt-21.2.5.0.0.211013################################# [100%]
2021-11-14 18:52:36 -0600: Post Installation steps in progress ...
Starting MS...
Importing snmp suscriber from compmon service...
Successfully imported snmp subscribers.
Installation SUCCESSFUL.
Done. Please Login as user dbmadmin.

I was able to list dbserver details and ran the validations again. The validations were successful and all checks passed.

 Using the note “After YUM upgrade imageinfo return Image activated and Image status blank (Doc ID 2518399.1)” as a guide, I updated image.id to replace failure with success:

# imageinfo -status
failure
# cd /opt/oracle.cellos/
# vi image.id
# imageinfo -status
success

Later, an Oracle engineer confirmed eligibility of the steps to fix the issue.
 
Have you encountered a similar problem while patching Exadata components? Drop me a line in the comments.

Don’t forget to sign up for more updates here.