Pythian Blog: Technical Track

How To Fix ASM Disk Header Status from "FORMER" To "MEMBER"

I made an error when creating an Oracle ASM disk group which resulted in not being able to drop or mount the disk group because the ASM header_status was FORMER. The following steps were performed to modify the ASM header_status from FORMER to MEMBER for version 12.1.0.2.0. Mounting and dropping disk group failed.
SQL> drop diskgroup ACFS_DATA;
 drop diskgroup ACFS_DATA
 *
 ERROR at line 1:
 ORA-15039: diskgroup not dropped
 ORA-15001: diskgroup "ACFS_DATA" does not exist or is not mounted
 
 
 SQL> alter diskgroup ACFS_DATA mount;
 alter diskgroup ACFS_DATA mount
 *
 ERROR at line 1:
 ORA-15032: not all alterations performed
 ORA-15017: diskgroup "ACFS_DATA" cannot be mounted
 ORA-15040: diskgroup is incomplete
 
 SQL> 
 
Review ASM disks. Note header_status is FORMER and group# is 0 (does not belong to group).
SQL> SELECT group_number group#, disk_number disk#, header_status header_, mode_status mode_, state, library, os_mb, total_mb, free_mb, label, path
  2 FROM V$ASM_DISK
  3 ORDER BY header_status desc, path
  4 ;
 
 GROUP# DISK# HEADER_ MODE_ STATE LIBRARY OS_MB TOTAL_MB FREE_MB LABEL PATH
 ------ ----- ---------- ---------- ------------ ---------------------------- ---------- ---------- ---------- --------------- ----------------------------------------
  0 1 FORMER ONLINE NORMAL System 153584 0 0 /dev/mapper/acfs_01p1
 
 22 rows selected.
 
Let's fix the issue.
Documentation used head -25 and might be better to grep for kfdhdb.hdrsts ### Use kfed to read device and confirm header_status KFDHDR_FORMER
$ kfed read '/dev/mapper/acfs_01p1' | head -25
 kfbh.endian: 1 ; 0x000: 0x01
 kfbh.hard: 130 ; 0x001: 0x82
 kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD
 kfbh.datfmt: 1 ; 0x003: 0x01
 kfbh.block.blk: 0 ; 0x004: blk=0
 kfbh.block.obj: 2147483648 ; 0x008: disk=0
 kfbh.check: 1999457674 ; 0x00c: 0x772d4d8a
 kfbh.fcn.base: 0 ; 0x010: 0x00000000
 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
 kfbh.spare1: 0 ; 0x018: 0x00000000
 kfbh.spare2: 0 ; 0x01c: 0x00000000
 kfdhdb.driver.provstr: ORCLDISK ; 0x000: length=8
 kfdhdb.driver.reserved[0]: 0 ; 0x008: 0x00000000
 kfdhdb.driver.reserved[1]: 0 ; 0x00c: 0x00000000
 kfdhdb.driver.reserved[2]: 0 ; 0x010: 0x00000000
 kfdhdb.driver.reserved[3]: 0 ; 0x014: 0x00000000
 kfdhdb.driver.reserved[4]: 0 ; 0x018: 0x00000000
 kfdhdb.driver.reserved[5]: 0 ; 0x01c: 0x00000000
 kfdhdb.compat: 202375168 ; 0x020: 0x0c100000
 kfdhdb.dsknum: 0 ; 0x024: 0x0000
 kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL
 **********************************************************************
 kfdhdb.hdrsts: 4 ; 0x027: KFDHDR_FORMER
 **********************************************************************
 kfdhdb.dskname: ACFS_DATA_0000 ; 0x028: length=14
 kfdhdb.grpname: ACFS_DATA ; 0x048: length=9
 kfdhdb.fgname: ACFS_DATA_0000 ; 0x068: length=14
 
 ### Backup device
 $ mkdir -p /tmp/asm
 $ dd if=/dev/mapper/acfs_01p1 of=/tmp/asm/ihotelp1_acfs.dmp bs=1048576 count=50
 50+0 records in
 50+0 records out
 52428800 bytes (52 MB) copied, 0.303025 s, 173 MB/s
 
 ### Create template for patching device
 $ kfed read /dev/mapper/acfs_01p1 > /tmp/asm/patch_acfs_01p1.txt
 
 ### Create copy of template for comparison
 $ cp patch_acfs_01p1.txt patch_acfs_01p1.orig
 
 ### Edit patch_acfs_01p1.txt and change "kfdhdb.hdrsts: 4 ; 0x027: KFDHDR_FORMER" to "kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER"
 
 ### Verify change.
 $ diff patch_acfs_01p1.txt patch_acfs_01p1.orig
 < kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER --- > kfdhdb.hdrsts: 4 ; 0x027: KFDHDR_FORMER
 
 ### Patch device acfs_01p1 using patch_acfs_01p1.txt
 $ kfed merge /dev/mapper/acfs_01p1 text=patch_acfs_01p1.txt
 
 ### Verify device acfs_01p1 is updated`
 $ kfed read /dev/mapper/acfs_01p1 | grep kfdhdb.hdrsts
 kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER
 $
 
Verify results using SQL
SQL> SELECT group_number group#, disk_number disk#, header_status header_, mode_status mode_, state, library, os_mb, total_mb, free_mb, label, path
  2 FROM V$ASM_DISK
  3 ORDER BY header_status desc, path
  4 ;
 
 GROUP# DISK# HEADER_ MODE_ STATE LIBRARY OS_MB TOTAL_MB FREE_MB LABEL PATH
 ------ ----- ---------- ---------- ------------ ---------------------------- ---------- ---------- ---------- --------------- ----------------------------------------
  0 1 MEMBER ONLINE NORMAL System 153584 0 0 /dev/mapper/acfs_01p1
 
 SQL> select name from v$asm_diskgroup order by 1;
 
 NAME
 ------------------------------------------------------------------------------------------
 ACFS_DATA
 
 SQL> alter diskgroup ACFS_DATA mount;
 
 Diskgroup altered.
 
 SQL> SELECT group_number group#, disk_number disk#, header_status header_, mode_status mode_, state, library, os_mb, total_mb, free_mb, label, path
  2 FROM V$ASM_DISK
  3 ORDER BY header_status desc, path
  4 ;
 
 GROUP# DISK# HEADER_ MODE_ STATE LIBRARY OS_MB TOTAL_MB FREE_MB LABEL PATH
 ------ ----- ---------- ---------- ------------ ---------------------------- ---------- ---------- ---------- --------------- ----------------------------------------
  5 0 MEMBER ONLINE NORMAL System 153584 153584 153500 /dev/mapper/acfs_01p1
 
 SQL> select name from v$asm_diskgroup order by 1;
 
 NAME
 ------------------------------------------------------------------------------------------
 ACFS_DATA
 
 SQL> 
 
 no rows selected
 
 SQL> 
 
Verify results using asmcmd.
$ asmcmd lsdg -g ACFS_DATA
 Inst_ID State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
  1 MOUNTED EXTERN N 512 4096 4194304 153584 153500 0 153500 0 N ACFS_DATA/
 
 $ asmcmd lsattr -l -G ACFS_DATA
 Name Value
 access_control.enabled FALSE
 access_control.umask 066
 au_size 4194304
 cell.smart_scan_capable FALSE
 compatible.advm 12.1.0.0.0
 compatible.asm 12.1.0.0.0
 compatible.rdbms 12.1.0.0.0
 content.check FALSE
 content.type data
 disk_repair_time 3.6h
 failgroup_repair_time 24.0h
 idp.boundary auto
 idp.type dynamic
 phys_meta_replicated true
 sector_size 512
 thin_provisioned FALSE
 $
 
Hopefully the information and demonstration will be useful should you encounter the same issues. Reference: ASM Corruption: Case #1: How To Fix The ASM Disk HEADER_STATUS From FORMER or PROVISIONED To MEMBER. (Doc ID 1448799.1)

No Comments Yet

Let us know what you think

Subscribe by email