ORA-17502: ksfdcre:4 Failed to create file +FRA

A weekend is about the start and got a call and one of our database is having issue. User is unable to connect and all their activity seems hung.

ISSUE : Instance Terminated and make Node2 inaccessible due to Oracle Sees FRA is Full.
But FRA is not Full ~ 2TB is available and db_recovery_file_dest_size=2.7TB. +FRA is ~18% Utilized only.

What i Did? Drop all existing restore point and restarted all node.

So i check the alert log on node 1 database.

 
*************************************
RVWR encountered an error when writing flashback database logs.
See error stack in alert log.
Since a Guarantee Restore Point was created,
RVWR cannot force to turn off flashback.
Therefore RVWR will terminate this instance.
*************************************
Fri Sep 16 05:17:28 2016
Errors in file /optware/orabase/grid/diag/rdbms/ckyclu3/CXXXLU31/trace/CXXXLU31_rvwr_41392.trc:
ORA-38701: Flashback database log 9 seq 5 thread 1: "+FRA"
ORA-17502: ksfdcre:4 Failed to create file +FRA
ORA-15041: diskgroup "FRA" space exhausted
USER (ospid: 41392): terminating the instance due to error 38701
Fri Sep 16 05:17:28 2016
System state dump requested by (instance=1, osid=41392 (RVWR)), summary=[abnormal instance termination].
System State dumped to trace file /optware/orabase/grid/diag/rdbms/ckyclu3/CXXXLU31/trace/CXXXLU31_diag_50122_20160916051728.trc
Fri Sep 16 05:17:30 2016
License high water mark = 81
Fri Sep 16 05:17:33 2016
Instance terminated by USER, pid = 41392
Fri Sep 16 05:17:34 2016
USER (ospid: 1359): terminating the instance
Fri Sep 16 05:17:34 2016
Instance terminated by USER, pid = 1359
Fri Sep 16 05:17:35 2016 

Continue reading

How To : Upgrade 11.2.0.2.0 to 12.1.0.1.0

it’s a little bit long time since my last post. though i have some notes on my activity i don’t have to update here.

Assumption :
SID : BPMPROD
old ORACLE_HOME = /opt/app/oracle/product/11.2.0.2/db_1
New ORACLE_HOME = /opt/app/oracle/product/12.1.0.1/dbhome_1

so today, i’m doing upgrade of oracle 11g to oracle 12c. Continue reading

Creating New Container Database

on my previous post we install oracle 12c software using silent mode. i always use silent mode since the server i’m using is located on other side of the world.

dbca -silent \
-createDatabase \
-templateName General_Purpose.dbc \
-gdbName ORA12C \
-sid ORA12C \
-SysPassword oracle12c \
-SystemPassword manager \
-datafileDestination /backup/ORA12CTESTING/db04/oradata \
-redoLogFileSize 500 \
-storageType FS \
-characterSet AL32UTF8 \
-NATIONALCHARACTERSET AL16UTF16 \
-totalMemory 1000 \
-createAsContainerDatabase true \
-emConfiguration DBEXPRESS  \
-registerWithDirService false 

Continue reading