Upgrading grid infrastructure from 12.1 to 12.2 in Exadata

1.2 Upgrade OPatch if OPatch is not already at the latest version:

 [oracle@exadatadb01 ~]$ unzip -q /software/27475857/Database/OPatch/12.2/12.2.0.1.13/p6880880_12*_Linux-x86-64.zip -d /u01/app/12.2.0.1/grid
 

1.3 Check that Oracle has a 10240 limits for the "soft stack" (if not, set it, log off and log on).

[root@exadatadb01~]$ dcli -g ~/dbs_group -l root grep stack /etc/security/limits.conf | grep soft
 exadatadb01: * soft stack 10240
 exadatadb02: * soft stack 10240
 exadatadb03: * soft stack 10240
 exadatadb04: * soft stack 10240
 

1.4 Verify that 1500 huge pages are available.

[oracle@exadatadb01~]$ dcli -g ~/dbs_group -l root grep -i huge /proc/meminfo
 ....
 AnonHugePages: 0 kB
 HugePages_Total: 200000
 HugePages_Free: 132171
 HugePages_Rsvd: 38338
 HugePages_Surp: 0
 Hugepagesize: 2048 kB
 ....
 

1.5 Fix the ntp failures in prerequisites check.

-- Verifying NTP daemon is synchronized with at least one external time source ... FAILED.

1.6 Run the prerequisites.

sudo su - oracle
 cd /u01/app/12.2.0.1/grid
 ./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /u01/app/12.1.0.2/grid -dest_crshome /u01/app/12.2.0.1/grid -dest_version 12.2.0.1 -fixup -verbose
 

2 Upgrade GI to 12.2

2.1 Set the ASM memory prerequisites.

sqlplus / as sysasm
 alter system set sga_max_size = 3G scope=spfile sid='*';
 alter system set sga_target = 3G scope=spfile sid='*';
 alter system set memory_target=0 sid='*' scope=spfile;
 alter system set memory_max_target=0 sid='*' scope=spfile /* required workaround */;
 alter system reset memory_max_target sid='*' scope=spfile;
 alter system set use_large_pages=true sid='*' scope=spfile /* 11.2.0.2 and later(Linux only) */;
 

2.2 Reset mis-count.

sudo su - oracle
 . oraenv <<< +ASM1
 crsctl unset css misscount
 

2.3 Check for any rebalance operation (there should be none).

sudo su - oracle
 . oraenv <<< +ASM1
 sqlplus / as sysasm
 select count(*) from gv$asm_operation;
 

2.4 Gridsetup - verify the response file.

[oracle@exadatadb01 ~]$ egrep -v "^#|^$" /tmp/giresponse.rsp | head -10
 oracle.install.responseFileVersion=/oracle/install/rspfmt_crsinstall_response_schema_v12.2.0
 INVENTORY_LOCATION=
 oracle.install.option=UPGRADE
 ORACLE_BASE=/u01/app/oracle
 oracle.install.asm.OSDBA=dba
 oracle.install.asm.OSOPER=dba
 oracle.install.asm.OSASM=dba
 oracle.install.crs.config.gpnp.scanName=
 oracle.install.crs.config.gpnp.scanPort=
 oracle.install.crs.config.ClusterConfiguration=
 [oracle@exadatadb01 ~]$
 

2.5 Gridsetup - run from exadatadb01.

sudo su - oracle
 cd /u01/app/12.2.0.1/grid
 ./gridSetup.sh -silent -responseFile /tmp/giresponse.rsp -J-Doracle.install.mgmtDB=false -J-Doracle.install.crs.enableRemoteGIMR=false -applyPSU /software/27475857/Database/12.2.0.1.0/12.2.0.1.180417GIRU/27850694
 

2.6 rootupgrade.sh on exadatadb01/2/3:

/u01/app/12.2.0.1/grid/rootupgrade.sh
 

2.7 rootupgrade.sh on exadatadb04, it will be the last one.

/u01/app/12.2.0.1/grid/rootupgrade.sh
 

2.8 Check if GI is relinked with RDS:

sudo su - oracle
 dcli -g /tmp/dbs_group -l oracle /u01/app/12.2.0.1/grid/bin/skgxpinfo
 
 => the command should return "rds" for each node
 

# If not, do it:

dcli -g /tmp/dbs_group -l oracle ORACLE_HOME=/u01/app/12.2.0.1/grid \ make -C /u01/app/12.2.0.1/grid/rdbms/lib -f ins_rdbms.mk ipc_rds ioracle
 

2.9 Check the status of the cluster and the installed version.

sudo su - oracle
 /u01/app/12.2.0.1/grid/bin/crsctl check cluster -all
 /u01/app/12.2.0.1/grid/bin/crsctl query crs activeversion
 

2.10 Modify ASM.

sudo su - oracle
 srvctl modify asm -count ALL
 

2.11 Update compatible.asm on the diskgroups.

$ . oraenv <<< +ASM1
 $ sqlplus / as sysasm
 SQL> select name, COMPATIBILITY from v$asm_diskgroup ;
 SQL> ALTER DISKGROUP DATA SET ATTRIBUTE 'compatible.asm' = '12.2.0.1.0';
 SQL> ALTER DISKGROUP DBFS_DG SET ATTRIBUTE 'compatible.asm' = '12.2.0.1.0';
 SQL> ALTER DISKGROUP RECO SET ATTRIBUTE 'compatible.asm' = '12.2.0.1.0';
 SQL> select name, COMPATIBILITY from v$asm_diskgroup ;
 

2.12 Update inventory.

$ . oraenv <<< +ASM1
 $ /u01/app/12.2.0.1/grid/oui/bin/runInstaller -ignoreSysPrereqs -updateNodeList ORACLE_HOME=/u01/app/12.2.0.1/grid "CLUSTER_NODES={exadatadb01,exadatadb02,exadatadb03,exadatadb04}" CRS=true LOCAL_NODE=exadatadb01
 

3 Known issues

3.1 The spfile was not being used by the ASM instances.

# If the command above doesn't return anything, which means the spfile is not being used. We need to set the spfile, restart the cluster if needed.

3.2 The password file of ASM instances was not in ASM diskgroup, you will see the failures in prerequisites check.

3.3 The patch 21255373 had been applied to GI, while when doing the prerequisites, it always reports the failures. According to Oracle support, this can be ignored. So the binary installation will have the option "-skipPrereqs".

3.4 The postscripts was not successful as the SCAN and SCAN_LISTENERS were not up.

3.5 Add the ASM instance entry into /etc/oratab.

For some reason, the ASM instance entry had been removed from /etc/oratab and never been added back. We need to add it manually to all nodes.