Pythian Blog: Technical Track

How to patch an exadata (part 4) - the rollback procedure

Quick links to Part 1 / Part 2 / Part 3 / Part 4 / Part 5 / Part 6
 
 

4: The Rollback Procedure

You will most likely never need to rollback any patch on an Exadata but it is interesting to know that it is possible and where to find the information in case of it happens one day; it is indeed possible to rollback any part of the patch. I list the procedure here but note that I have never tested it. Note that for the cells, it is only possible to rollback a successful update. However, if you face a failed cell update, read my first advice again, check the troubleshooting section of this blog, the known issues of the patch readme and if you found nothing there, open a Sev 1 SR to Oracle Support. I will not talk about the GI and Database OH rollback procedure here as there's nothing specific here, it is the same as with non-Exadata GI and DB OH.

 

4.1 - Cell Rollback

We can read in the readme that it is only possible to rollback a successfully-updated Exadata Cells. Cells with incomplete or failed updates cannot be rolled back. If you really want to rollback a successfully patched cell, here is the procedure you will find in the Readme section " 2.3 Rolling Back Successfully Updated Exadata Cells" :
  • Check the version cells will be rolled back to and the flashCacheMode setting with the following commands :
[root@myclusterdb01 ~]# dcli -l root -g cell_group imageinfo -ver -inactive
 [root@myclusterdb01 ~]# dcli -l root -g cell_group cellcli -e 'list cell attributes flashCacheMode'
 
Cells being rolled back to releases earlier than release 11.2.3.2.0 with write back flash cache enabled need to be converted manually back to write through flash cache before being rolled back. Disable write back flash cache using the script in My Oracle Support note 1500257.1. Cells being rolled back to release 11.2.3.2.0 or later retain the flash cache mode that is currently set.
  • Check the prerequisites using the following command:
[root@myclusterdb01 ~]# ./patchmgr -cells cell_group -rollback_check_prereq -rolling
 [root@myclusterdb01 ~]# ./patchmgr -cells cell_group -rollback -rolling
 
  • Clean up the cells using the -cleanup option to clean up all the temporary update or rollback files on the cells. This option cleans the stale update and rollback states as well as cleaning up to 1.5 GB of disk space on the cells. Use this option before retrying a halted or failed run of the patchmgr utility :
[root@myclusterdb01 ~]# ./patchmgr -cells cell_group -cleanup
 

 

4.2 - DB nodes Rollback

Here is the procedure that can be found in the readme to rollback a database node.
  • Check the version of each DB node before the rollback :
[root@myclusterdb01 ~]# dcli -g ~/dbs_group -l root imageinfo -ver
 
  • Umount the NFS :
You can generate all the umount commands with this command:
df -t nfs | awk '{if ($NF ~ /^\//){print "umount " $NF}}'
 
  • Rollback the patch (launch it from the cel01 server) :
[root@myclustercel01 ~]# cd /tmp/dbserver_patch_5.161110
 [root@myclustercel01 dbserver_patch_5.161110]# ./patchmgr -dbnodes ~/dbs_group -rollback -iso_repo /tmp/p24669306_121233_Linux-x86-64.zip -target_version <previous the='' of='' version='' installed='' patch=''> -rolling
 
  • Check the version of each DB node after the rollback:
[root@myclusterdb01 ~]# dcli -g ~/dbs_group -l root imageinfo -ver
 

 

4.3 - IB Switches Rollback

As per the documentation, it is only possible to rollback an IB Switch to version 2.1.6-2 Be sure to be connected to the server myclusterdb01 (a server where the SSH keys are deployed to the IB switches)
  • Check the current version installed on the IB switches :
[root@myclusterdb01 ~]# dcli -g ~/ib_group -l root version | grep -i version
 
Downgrade the version to the only possible version : 2.1.6-2
[root@myclusterdb01 ~]# cd /patches/OCT2016_bundle_patch/24436624/Infrastructure/12.1.2.3.3/ExadataStorageServer_InfiniBandSwitch/patch_12.1.2.3.3.161013/
 [root@myclusterdb01 ~]# ./patchmgr -ibswitches ~/ib_group -downgrade -ibswitch_precheck
 [root@myclusterdb01 ~]# ./patchmgr -ibswitches ~/ib_group -downgrade
 
  • Check the versions on the IB switches after the rollback:
[root@myclusterdb01 ~]# dcli -g ~/ib_group -l root version | grep -i version
 
 
Quick links to Part 1 / Part 2 / Part 3 / Part 4 / Part 5 / Part 6
 

No Comments Yet

Let us know what you think

Subscribe by email