Patch 15887843 MERGE REQUEST ON TOP OF 11.2.3.2.0 FOR BUGS 15841041, 15874381

Patching time again..

Patch 15887843 MERGE REQUEST ON TOP OF 11.2.3.2.0 FOR BUGS 15841041, 15874381
Below are the list of bug fixed on this release.

========== Bug fixes or Diagnostics included in this ONEOFF ===========
Bug Fixes:
        11903713:CELL-2628 DURING LOOP TEST OF CELLCLI LIST QUERIES
        14337259:ASM ARB0 HANG ON REBALANCE IO THAT IS NOT ALIGNED IF BLOCK DIRTY IN FLASH CACHE
        14540423:ORA-7445 [__INTEL_NEW_MEMCPY() CAUSED BY INCORRECT MANAGEMENT OF CHAINED ROWS
        14763896:TOO MANY FILE SYSTEM IOS MAY CAUSE CONFINEMENT DETECTION ON SYSTEM DISKS
        14798635:DOM CONFINEMENT TRIGGERED ON PARTNER CELLS WITH NO WORKLOAD
        14823493:DO NOT POWER CYCLE FOR IO HANG ON WRITE BACK FLASHCACHE
        14841379:SAFEFILE NEEDS FSYNC SUPPORT TO COMMIT CHANGES
        15834287:IF CANNOT GET LOCK ON GRIDDISK LIST ON FLASH DISK, DURING IO HANG DONT REBOOT
        14828059:ORA-1578 REPORTED IN SUPERCLUSTER WITH EXADATA


Copy all file to all storage cell. i’ll be using a DB node to upload all files.

[root@exa2db01 ~]#  dcli -l root -g listcell -f cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin -d /tmp
[root@exa2db01 ~]#  dcli -l root -g listcell ls -l /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
 
exa2cel01: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel02: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel03: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel04: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel05: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel06: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel07: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel08: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel09: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel10: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel11: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel12: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel13: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
exa2cel14: -rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin

1. By default, ASM drops a disk shortly after it is taken offline; however, you can set the DISK_REPAIR_TIME attribute
to prevent this operation by specifying a time interval to repair the disk and bring it back online.
The default DISK_REPAIR_TIME attribute value of 3.6h should be adequate for most environments

To check repair times for all mounted disk groups – log into the ASM instance and perform the following query

set lines 200
col VALUE format a30
select dg.name,a.value from v$asm_diskgroup
dg, v$asm_attribute a where dg.group_number=a.group_number and
a.name='disk_repair_time'; 

If you need to offline the ASM disks for more than the default time of 3.6 hours then adjust the parameter by issuing the command below as an example

ALTER DISKGROUP DATA SET ATTRIBUTE 'DISK_REPAIR_TIME'='8.5H';


2. Next you will need to check if ASM will be OK if the grid disks go OFFLINE.

The following command should return ‘Yes’ for the grid disks being listed:

cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome 

3. If one or more disks return asmdeactivationoutcome=’No’, you should wait for some time and repeat step #2. Once all disks return return asmdeactivationoutcome=’Yes’, you can proceed with taking the griddisk offline in step #4.

Note: Taking the storage server offline when one or more grid disks return asmdeactivationoutcome=’No’ will cause Oracle ASM to dismount the affected disk group,
causing the databases to shut down abruptly.

4. Run cellcli command to Inactivate all grid disks on the cell you wish to power down/reboot:

cellcli -e alter griddisk all inactive

* Please note – This action could take 10 minutes or longer depending on activity.
It is very important to make sure you were able to offline all the disks successfully before shutting down the cell services.
Inactivating the grid disks will automatically OFFLINE the disks in the ASM instance.

5. Confirm that the griddisks are now offline by performing the following actions:

(a) Execute the command below and the output should show asmmodestatus=UNUSED and asmdeactivationoutcome=Yes
for all griddisks once the disks are offline in ASM. Only then is it safe to proceed with shutting down or restarting the cell:

cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome

( there has also been a reported case of asmmodestatus= OFFLINE: Means Oracle ASM has taken this grid disk offline.
This status is also fine and can proceed with remaining instructions)

(b) List the griddisks to confirm all now show offline:

cellcli -e list griddisk 

6. Apply the patch .

Please verify that there is at least around 900 MB of space available in
/opt/oracle.  If there is not enough, please cleanup files.
Contents of /opt/oracle/cell/cellsrv/deploy/config/metrics can be deleted
if necessary as they are one of the larger consumers of space.
 
b.3) For each cell, do the following:
 
% /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin --doall --force
 
[root@exa2cel14 tmp]# ls -l /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
-rwxr-xr-x 1 root root 242793674 Dec  4 00:01 /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin
[root@exa2cel14 tmp]# /tmp/cell-11.2.3.2.0.15887843_LINUX.X64_121117-1-rpm.bin --doall --force
UnZipSFX 5.51 of 22 May 2004, by Info-ZIP (http://www.info-zip.org).
  inflating: cell-11.2.3.2.0.15887843_LINUX.X64_121117-1.x86_64.rpm
  inflating: jdk-1_5_0_15-linux-amd64.rpm
JDK already installed
Installing  cell  software:
Upgrade to version 11.2.3.2.0.15887843_LINUX.X64_121117-1
Preparing...                ########################################### [100%]
Pre Installation steps in progress ...
Trying to upgrade from version 11.2.3.2.0_LINUX.X64_120713 to version 11.2.3.2.0.15887843_LINUX.X64_121117!
Cellsrv is shutdown ..
   1:cell                   ########################################### [100%]
Post Installation steps in progress ...
Set cellusers group for /opt/oracle/cell11.2.3.2.0.15887843_LINUX.X64_121117/cellsrv/deploy/log directory
Set 775 permissions for /opt/oracle/cell11.2.3.2.0.15887843_LINUX.X64_121117/cellsrv/deploy/log directory
/
/
New CELLSRV started at time Tue Dec  4 00:50:41 UTC 2012
WARNING: Using the current shell as root to restart cell services.
Restart the cell services using a new shell.
Upgrade was successful. Uninstall cell-11.2.3.2.0_LINUX.X64_120713-1
[root@exa2cel14 tmp]#

7. Now we need to reactive the griddisks, make sure to create a new ssh connection

cellcli -e alter griddisk all active 

8. Issue the command below and all disks should show ‘active’:

cellcli -e list griddisk

9. Verify grid disk status:

(a) Verify all grid disks have been successfully put online using the following command:

cellcli -e list griddisk attributes name, asmmodestatus

(b) Wait until asmmodestatus is ONLINE for all grid disks. Each disk will go to a ‘SYNCING’ state first then ‘ONLINE’. The following is an example of the output:
(c) Oracle ASM synchronization is only complete when all grid disks show asmmodestatus=ONLINE.

( Please note: this operation uses Fast Mirror Resync operation – which does not trigger an ASM rebalance.
The Resync operation restores only the extents that would have been written while the disk was offline.)

its time to restart another Cell… 13 to go.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s