Please see my other blog for Oracle EBusiness Suite Posts - EBMentors

Search This Blog

Note: All the posts are based on practical approach avoiding lengthy theory. All have been tested on some development servers. Please don’t test any post on production servers until you are sure.

Monday, November 24, 2014

Exadata: Locating Oracle Cluster Registry and Voting Disks

On Exadata, Oracle has elected to store the Oracle Cluster Registry (OCR) and voting disks on Oracle ASM disk groups, mapped to Exadata storage server grid disks. The OCR and voting disks are typically stored in the DBFS_DG ASM disk group and associated grid disks.


If you want to determine the location and configuration of your Oracle Cluster Registry and voting disks on Oracle Exadata, perform the below steps.

1- The Oracle 11gR2 Cluster Registry (OCR) and voting disks are stored on Oracle ASM disk groups on Exadata. In an Oracle 11gR2 Grid Infrastructure environment running Oracle Clusterware, regardless of whether you are on Exadata or not, the OCR location is specified in /etc/oracle/ocr.loc. Begin by logging in to a compute node and checking the contents of your ocr.loc file:

[root@pk3-iub-rp-od01 /]# locate ocr.loc
/etc/oracle/ocr.loc
/etc/oracle/ocr.loc.orig

[root@pk3-iub-rp-od01 /]# cat /etc/oracle/ocr.loc
ocrconfig_loc=+DBFS_DG
local_only=FALSE

As you can see, the OCR is contained in the DBFS_DG ASM disk group. Starting with Oracle 11gR2, you can store your OCR inside an ASM disk group, and on Exadata this is the default and preferred. 

2- You can run ocrcheck from the Grid Infrastructure Oracle software location to examine additional details about your OCR:
[root@pk3-iub-rp-od01 /]# /u01/app/11.2.0.4/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3308
         Available space (kbytes) :     258812
         ID                       : 1348892576
         Device/File Name         :   +DBFS_DG
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

3- As with non-Exadata Grid Infrastructure installations, Oracle automatically makes a backup of the OCR every four hours, and you can find the backup location and backup details using ocrconfig –showbackup:

[root@pk3-iub-rp-od01 /]# /u01/app/11.2.0.4/grid/bin/ocrconfig -showbackup

pk3-iub-rp-od01     2014/06/16 11:37:20     /u01/app/11.2.0.4/grid/cdata/pk3-iub-cluster/backup00.ocr
pk3-iub-rp-od01     2014/06/16 07:37:20     /u01/app/11.2.0.4/grid/cdata/pk3-iub-cluster/backup01.ocr
pk3-iub-rp-od01     2014/06/16 03:37:19     /u01/app/11.2.0.4/grid/cdata/pk3-iub-cluster/backup02.ocr
pk3-iub-rp-od01     2014/06/15 03:37:18     /u01/app/11.2.0.4/grid/cdata/pk3-iub-cluster/day.ocr
pk3-iub-rp-od01     2014/06/09 03:37:08     /u01/app/11.2.0.4/grid/cdata/pk3-iub-cluster/week.ocr
pk3-iub-rp-od01     2013/12/16 16:49:40     /u01/app/11.2.0.4/grid/cdata/pk3-iub-cluster/backup_20131216_164940.ocr
pk3-iub-rp-od01     2013/12/16 16:49:39     /u01/app/11.2.0.4/grid/cdata/pk3-iub-cluster/backup_20131216_164939.ocr

4- On Exadata, the clusterware voting disks are also stored in an Oracle ASM disk group, as indicated:
[root@pk3-iub-rp-od01 /]# /u01/app/11.2.0.4/grid/bin/crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   65e514063aa24f73bf38ccbbb1ef06da (o/20.168.166.13/DBFS_DG_CD_02_pk3_iub_cel_es01) [DBFS_DG]
 2. ONLINE   9ff55be6aa614f99bfd97527fec7e7ac (o/20.168.166.14/DBFS_DG_CD_02_pk3_iub_cel_es02) [DBFS_DG]
 3. ONLINE   102134e2dfda4f26bfe3be5b2805d9bc (o/20.168.166.15/DBFS_DG_CD_02_pk3_iub_cel_es03) [DBFS_DG]
Located 3 voting disk(s).

You can see that Exadata builds three copies of the voting disk, each stored in the DBFS_DG ASM disk group, with copies on each of your three Exadata Storage Servers.

5- Get the information about associated grid disks

CellCLI> list griddisk where name like 'DBFS_DG_.*' attributes name,cellDisk,size
         DBFS_DG_CD_02_pk3_iub_cel_es01  CD_02_pk3_iub_cel_es01  29.125G
         DBFS_DG_CD_03_pk3_iub_cel_es01  CD_03_pk3_iub_cel_es01  29.125G
         DBFS_DG_CD_04_pk3_iub_cel_es01  CD_04_pk3_iub_cel_es01  29.125G
         DBFS_DG_CD_05_pk3_iub_cel_es01  CD_05_pk3_iub_cel_es01  29.125G
         DBFS_DG_CD_06_pk3_iub_cel_es01  CD_06_pk3_iub_cel_es01  29.125G
         DBFS_DG_CD_07_pk3_iub_cel_es01  CD_07_pk3_iub_cel_es01  29.125G
         DBFS_DG_CD_08_pk3_iub_cel_es01  CD_08_pk3_iub_cel_es01  29.125G
         DBFS_DG_CD_09_pk3_iub_cel_es01  CD_09_pk3_iub_cel_es01  29.125G
         DBFS_DG_CD_10_pk3_iub_cel_es01  CD_10_pk3_iub_cel_es01  29.125G
         DBFS_DG_CD_11_pk3_iub_cel_es01  CD_11_pk3_iub_cel_es01  29.125G
 

No comments: