RAC Attack - Oracle Cluster Database at Home/Install Grid Infrastructure (Shared Filesystem)

From Wikibooks, open books for an open world
Jump to: navigation, search


Several NFS appliances and big-iron cluster filesystems are very common in large cluster database deployments. We will use OCFS2 here to practice 11gR2 RAC with a filesystem.



  1. As the oracle user, launch the grid installer. At the first screen, choose Install and Configure Grid Infrastructure for a Cluster and click NEXT.

    [oracle@collabn1 ~]$ /mnt/cdrom*/grid/runInstaller
    
    RA-grid-install-splash.png


    RA-grid-install-option.png
  2. Choose Advanced Installation and click NEXT.

    RA-grid-install-type.png
  3. Accept the default language (English) and choose NEXT.

    RA-grid-install-language.png
  4. Name the cluster collab and make sure that the SCAN name is collab-scan with port 1521, then click NEXT.

    RA-grid-install-gpnp.png
  5. Add node collabn2 with VIP collabn2-vip and choose NEXT to validate the cluster configuration.

    RA-grid-install-nodes.png
  6. Verify that eth0 on subnet 192.168.78.0 is PUBLIC and that eth1 on subnet 172.16.100.0 is PRIVATE, then click NEXT.

    RA-grid-install-network.png
  7. Choose to store the Clusterware Files in Shared File System and choose NEXT.

    RA-grid-install-storage-shared-fs.png
  8. For the OCR, choose External Redundancy and type the path /u61/cluster/ocr. (This is the NFS location from the BUG WORKAROUND.) Click NEXT to continue.

    RA-grid-install-storage-ocr.png
  9. For the Voting Disk, do the same – choose External Redundancy and type the path /u61/cluster/vdsk. (Again, this is the NFS location from the BUG WORKAROUND.) Click NEXT to continue.

    RA-grid-install-storage-voting.png
  10. Do not use IPMI. Click NEXT.

    RA-grid-install-ipmi.png
  11. Set the OSDBA group to asmdba, the OSOPER group to asmoper and the OSASM group to asmadmin. Then click NEXT.

    RA-grid-install-groups.png
  12. Accept the ORACLE_BASE location of /u01/app/oracle and use the ORACLE_HOME location of /u01/grid/oracle/product/11.2.0/grid_1. Then click NEXT.

    RA-grid-install-locations.png
  13. Accept the default inventory location and choose NEXT

    RA-grid-install-inventory.png
  14. The prerequisite checks will execute. A warning will be issued saying that three checks failed: physical memory, swap size and network time protocol. Click the CHECK BOX to Ignore All, then click NEXT.

    RA-grid-install-checks.png
  15. SAVE a response file called grid.rsp in the oracle user home directory. Then click FINISH to install grid infrastructure.

    RA-grid-install-save-rsp.png


    RA-grid-install-inprogress.png
  16. When prompted, open a terminal as the root user and run the two root.sh scripts. Make sure to run BOTH SCRIPTS on BOTH NODES!

    RA-grid-install-scripts.png
    [oracle@collabn1 ~]$ su -
    Password: racattack
     
    [root@collabn1 ~]# /u01/app/oraInventory/orainstRoot.sh
    Changing permissions of /u01/app/oraInventory.
    Adding read,write permissions for group.
    Removing read,write,execute permissions for world.
     
    Changing groupname of /u01/app/oraInventory to oinstall.
    The execution of the script is complete.
     
    [root@collabn1 ~]# /u01/grid/oracle/product/11.2.0/grid_1/root.sh
    Running Oracle 11g root.sh script...
     
    The following environment variables are set as:
        ORACLE_OWNER= oracle
        ORACLE_HOME=  /u01/grid/oracle/product/11.2.0/grid_1
     
    Enter the full pathname of the local bin directory: [/usr/local/bin]: /usr/bin
       Copying dbhome to /usr/bin ...
       Copying oraenv to /usr/bin ...
       Copying coraenv to /usr/bin ...
     
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2011-03-30 16:49:13: Parsing the host name
    2011-03-30 16:49:13: Checking for super user privileges
    2011-03-30 16:49:13: User has super user privileges
    Using configuration parameter file: /u01/grid/oracle/product/11.2.0/grid_1/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
      root wallet
      root wallet cert
      root cert export
      peer wallet
      profile reader wallet
      pa wallet
      peer wallet keys
      pa wallet keys
      peer cert request
      pa cert request
      peer cert
      pa cert
      peer root cert TP
      profile reader root cert TP
      pa root cert TP
      peer pa cert TP
      pa peer cert TP
      profile reader pa cert TP
      profile reader peer cert TP
      peer user cert
      pa user cert
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-2672: Attempting to start 'ora.gipcd' on 'collabn1'
    CRS-2672: Attempting to start 'ora.mdnsd' on 'collabn1'
    CRS-2676: Start of 'ora.mdnsd' on 'collabn1' succeeded
    CRS-2676: Start of 'ora.gipcd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'collabn1'
    CRS-2676: Start of 'ora.gpnpd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'collabn1'
    CRS-2676: Start of 'ora.cssdmonitor' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'collabn1'
    CRS-2672: Attempting to start 'ora.diskmon' on 'collabn1'
    CRS-2676: Start of 'ora.diskmon' on 'collabn1' succeeded
    CRS-2676: Start of 'ora.cssd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'collabn1'
    CRS-2676: Start of 'ora.ctssd' on 'collabn1' succeeded
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    CRS-2672: Attempting to start 'ora.crsd' on 'collabn1'
    CRS-2676: Start of 'ora.crsd' on 'collabn1' succeeded
    Now formatting voting disk: /u61/cluster/vdsk.
    CRS-4603: Successful addition of voting disk /u61/cluster/vdsk.
    ##  STATE    File Universal Id                File Name Disk group
    --  -----    -----------------                --------- ---------
     1. ONLINE   59f9f62cc0f44f35bff552ced06e35ee (/u61/cluster/vdsk) []
    Located 1 voting disk(s).
    CRS-2673: Attempting to stop 'ora.crsd' on 'collabn1'
    CRS-2677: Stop of 'ora.crsd' on 'collabn1' succeeded
    CRS-2673: Attempting to stop 'ora.asm' on 'collabn1'
    CRS-2677: Stop of 'ora.asm' on 'collabn1' succeeded
    CRS-2673: Attempting to stop 'ora.ctssd' on 'collabn1'
    CRS-2677: Stop of 'ora.ctssd' on 'collabn1' succeeded
    CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'collabn1'
    CRS-2677: Stop of 'ora.cssdmonitor' on 'collabn1' succeeded
    CRS-2673: Attempting to stop 'ora.cssd' on 'collabn1'
    CRS-2677: Stop of 'ora.cssd' on 'collabn1' succeeded
    CRS-2673: Attempting to stop 'ora.gpnpd' on 'collabn1'
    CRS-2677: Stop of 'ora.gpnpd' on 'collabn1' succeeded
    CRS-2673: Attempting to stop 'ora.gipcd' on 'collabn1'
    CRS-2677: Stop of 'ora.gipcd' on 'collabn1' succeeded
    CRS-2673: Attempting to stop 'ora.mdnsd' on 'collabn1'
    CRS-2677: Stop of 'ora.mdnsd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.mdnsd' on 'collabn1'
    CRS-2676: Start of 'ora.mdnsd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.gipcd' on 'collabn1'
    CRS-2676: Start of 'ora.gipcd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'collabn1'
    CRS-2676: Start of 'ora.gpnpd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'collabn1'
    CRS-2676: Start of 'ora.cssdmonitor' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'collabn1'
    CRS-2672: Attempting to start 'ora.diskmon' on 'collabn1'
    CRS-2676: Start of 'ora.diskmon' on 'collabn1' succeeded
    CRS-2676: Start of 'ora.cssd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'collabn1'
    CRS-2676: Start of 'ora.ctssd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'collabn1'
    CRS-2676: Start of 'ora.asm' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.crsd' on 'collabn1'
    CRS-2676: Start of 'ora.crsd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.evmd' on 'collabn1'
    CRS-2676: Start of 'ora.evmd' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'collabn1'
    CRS-2676: Start of 'ora.asm' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.DATA.dg' on 'collabn1'
    CRS-2676: Start of 'ora.DATA.dg' on 'collabn1' succeeded
    CRS-2672: Attempting to start 'ora.registry.acfs' on 'collabn1'
    CRS-2676: Start of 'ora.registry.acfs' on 'collabn1' succeeded
     
    collabn1     2011/03/30 16:59:35     /u01/grid/oracle/product/11.2.0/grid_1/cdata/collabn1/backup_20110330_165935.olr
    Preparing packages for installation...
    cvuqdisk-1.0.7-1
    Configure Oracle Grid Infrastructure for a Cluster ... succeeded
    Updating inventory properties for clusterware
    Starting Oracle Universal Installer...
     
    Checking swap space: must be greater than 500 MB.   Actual 968 MB    Passed
    The inventory pointer is located at /etc/oraInst.loc
    The inventory is located at /u01/app/oraInventory
    'UpdateNodeList' was successful.
    


    [root@collabn1 ~]# ssh collabn2
    root@collabn2's password: racattack
    -bash: oraenv: No such file or directory
    [root@collabn2 ~]# /u01/app/oraInventory/orainstRoot.sh
    Changing permissions of /u01/app/oraInventory.
    Adding read,write permissions for group.
    Removing read,write,execute permissions for world.
     
    Changing groupname of /u01/app/oraInventory to oinstall.
    The execution of the script is complete.
     
    [root@collabn2 ~]# /u01/grid/oracle/product/11.2.0/grid_1/root.sh
    Running Oracle 11g root.sh script...
     
    The following environment variables are set as:
        ORACLE_OWNER= oracle
        ORACLE_HOME=  /u01/grid/oracle/product/11.2.0/grid_1
     
    Enter the full pathname of the local bin directory: [/usr/local/bin]: /usr/bin
       Copying dbhome to /usr/bin ...
       Copying oraenv to /usr/bin ...
       Copying coraenv to /usr/bin ...
     
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2011-03-30 17:04:26: Parsing the host name
    2011-03-30 17:04:26: Checking for super user privileges
    2011-03-30 17:04:26: User has super user privileges
    Using configuration parameter file: /u01/grid/oracle/product/11.2.0/grid_1/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node collabn1, number 1, and is terminating
    An active cluster was found during exclusive startup, restarting to join the cluster
    CRS-2672: Attempting to start 'ora.mdnsd' on 'collabn2'
    CRS-2676: Start of 'ora.mdnsd' on 'collabn2' succeeded
    CRS-2672: Attempting to start 'ora.gipcd' on 'collabn2'
    CRS-2676: Start of 'ora.gipcd' on 'collabn2' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'collabn2'
    CRS-2676: Start of 'ora.gpnpd' on 'collabn2' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'collabn2'
    CRS-2676: Start of 'ora.cssdmonitor' on 'collabn2' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'collabn2'
    CRS-2672: Attempting to start 'ora.diskmon' on 'collabn2'
    CRS-2676: Start of 'ora.diskmon' on 'collabn2' succeeded
    CRS-2676: Start of 'ora.cssd' on 'collabn2' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'collabn2'
    CRS-2676: Start of 'ora.ctssd' on 'collabn2' succeeded
    CRS-2672: Attempting to start 'ora.drivers.acfs' on 'collabn2'
    CRS-2676: Start of 'ora.drivers.acfs' on 'collabn2' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'collabn2'
    CRS-2676: Start of 'ora.asm' on 'collabn2' succeeded
    CRS-2672: Attempting to start 'ora.crsd' on 'collabn2'
    CRS-2676: Start of 'ora.crsd' on 'collabn2' succeeded
    CRS-2672: Attempting to start 'ora.evmd' on 'collabn2'
    CRS-2676: Start of 'ora.evmd' on 'collabn2' succeeded
     
    collabn2     2011/03/30 17:12:32     /u01/grid/oracle/product/11.2.0/grid_1/cdata/collabn2/backup_20110330_171232.olr
    Preparing packages for installation...
    cvuqdisk-1.0.7-1
    Configure Oracle Grid Infrastructure for a Cluster ... succeeded
    Updating inventory properties for clusterware
    Starting Oracle Universal Installer...
     
    Checking swap space: must be greater than 500 MB.   Actual 1205 MB    Passed
    The inventory pointer is located at /etc/oraInst.loc
    The inventory is located at /u01/app/oraInventory
    'UpdateNodeList' was successful.
    


  17. After running both scripts, return to the installer window and click OK to continue running configuration assistants.

    RA-grid-install-inprogress-post-scripts.png
  18. The Cluster Verification Utility will fail because NTP is not running. If you want to, check the error message at the very end of the logfile. Then click OK to close the messagebox and click NEXT to continue.

    RA-grid-install-cvu-failed.png
  19. You should now see the final screen! Click CLOSE to exit the installer.

    RA-grid-install-finished.png