Database fails to start with ORA-03113: end-of-file on communication channel error in RSA Identity Governance & Lifecycle
3 years ago
Originally Published: 2019-07-03
Article Number
000040941
Applies To
RSA Product Set: Identity Governance & Lifecycle



 
Issue
The RSA Identity Governance & Lifecycle database fails to start with the following errors:
  
$ acm startdb 
PRCR-1079 : Failed to start resource ora.avdb.db 
CRS-5017: The resource action "ora.avdb.db start" encountered the following error: 
ORA-03113: end-of-file on communication channel 
Process ID: 6353 
Session ID: 100 Serial number: 47767 
. For details refer to "(:CLSN00107:)" in "/u01/app/oracle/diag/crs/nlxlum062/crs/trace/ohasd_oraagent_oracle.trc". 

CRS-2674: Start of 'ora.avdb.db' on 'nlxlum062' failed 
Database is not running.

No changes have been made to the system.

The file /u01/app/oracle/diag/crs/nlxlum062/crs/trace/ohasd_oraagent_oracle.trc referred to in the error message shows the following messages: 
019-05-02 16:23:02.403474 :CLSDYNAM:199223040: [ora.avdb.db]{0:0:778} [start] 
(:CLSN00107:) clsn_agent::start } 
2019-05-02 16:25:26.540819 :CLSDYNAM:199223040: [ora.avdb.db]{0:0:821} [start] 
(:CLSN00107:) clsn_agent::start {
2019-05-02 18:42:07.245091 :CLSDYNAM:1732069120: [ora.avdb.db]{0:0:163} [start] 
(:CLSN00107:) clsn_agent::start } 
2019-05-02 18:42:07.246057 :CLSDYNAM:1732069120: [ora.avdb.db]{0:0:163} [check] 
(:CLSN00007:)DbAgent::checkCbk failed gimh state 0(Abnormal Termination) 
2019-05-02 18:42:07.246355 :CLSDYNAM:1732069120: [ora.avdb.db]{0:0:163} [check] 
(:CLSN00007:)InstAgent::checkState return unplanned offline

The Oracle alert log /u01/app/oracle/diag/rdbms/avdb/AVDB/trace/alert_AVDB.log has the following errors:  
Line 818460: ORA-19815: WARNING: db_recovery_file_dest_size of 107374182400 bytes is 99.87% used, 
and has 135806976 remaining bytes available. 
Line 818475: Errors in file /u01/app/oracle/diag/rdbms/avdb/AVDB/trace/AVDB_arc3_29567.trc: 
Line 818476: ORA-19809: limit exceeded for recovery files 
Line 818477: ORA-19804: cannot reclaim 729022976 bytes disk space from 107374182400 limit 
Line 818478: ARC3: Error 19809 Creating archive log file to 
'/u01/app/oracle/fast_recovery_area/AVDB/archivelog/2019_05_01/o1_mf_1_4798_%u_.arc' 
Line 818480: Errors in file /u01/app/oracle/diag/rdbms/avdb/AVDB/trace/AVDB_arc0_29561.trc: 
Line 818481: ORA-19815: WARNING: db_recovery_file_dest_size of 107374182400 bytes is 99.87% used, 
and has 135806976 remaining bytes available. 
Cause
Oracle archived redo logging is enabled and the system has run out of space for writing the archived redo logs. 

 

 


 
Resolution
Please note the supported backup method for RSA Identity Governance & Lifecycle is Oracle's database export utility expdb.  That is because much of the database activity is temporary and does not need to be restored. For example, during a collection run, multiple temporary tables are created where data is inserted and then deleted. It is temporary work that never needs to be recovered. Enabling archived redo logging will log all of these inserts and deletes which will slow down performance and increase disk space usage.

The resolution in this case would be to disable archived redo logging. Please contact your DBA for assistance with this resolution.


 
 
Workaround
If your DBA requires archive log mode to be enabled, then the problem needs to be resolved another way.

The Oracle parameter DB_RECOVERY_FILE_DEST_SIZE is used to limit the amount of disk space used for writing archived redo logs.  Running out of disk space can cause serious and unexpected consequences. This parameter helps avert that possibility by stopping the process before it actually physically fills up the disk.

One way to resolve this error is to add more disk space and then increase the size of the DB_RECOVERY_FILE_DEST_SIZE parameter to allow Oracle to use the additional disk space for logging the archived redo logs.

If you need assistance on this error, please contact your DBA.
 

Related Articles

Trending Articles

Don't see what you're looking for?