SP4 installation fails and automatic restoration breaks the replication.
2 years ago
Originally Published: 2010-12-30
Article Number
000050897
Applies To
SecurID Appliance 3.0.3.5
SecurID Appliance A130 3.0.4
capture_error => ORA-01013: user requested cancel of current operation
Issue
SP4 installation fails and automatic restoration breaks the replication.
SP4 installation fails and automatic restoration kicks off and restores to previous version 3.0.3.5
On Operations console replication status indicates need action under status.

When replication status is verified by below command it shows "ABORTED" for certain replication components:

./rsautil manage-database -a exec-sql -U com.rsa.replication.admin -f diagnostics/primary_replication_status.sql

Enter Master password: *********

 

 Manage Database ims-2.0.3-build20100922141135

 Copyright (C) 2008 RSA Security Inc. All rights reserved.

 

 %% Running at: authapp1.mgic.net:[r8b4e05i] %%

 

 ===========================================

 %             Execute SQL                %

 ===========================================

 Are you sure you want to execute the SQL file [diagnostics/primary_replication_status.sql] ? (Y/N): y

.

.SQL*Plus: Release 10.2.0.4.0 - Production on Wed Dec 29 14:27:05 2010

.

.Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.

.

.

.Connected to:

.Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production

.With the Partitioning, Data Mining and Real Application Testing options

.

.REPLICATION_CAPTURE_ERROR

14:27:06 : capture_name => RSA_STREAMS_CAPTURE

.capture_status => ABORTED

.capture_state =>

.capture_error => ORA-01013: user requested cancel of current operation

.capture_time =>

.used_arc_log_size => 2945

.max_arc_log_size => 102400

.min_arc_log_time => 21-12-2010 04:06:01 AM

.local_utc_time => 29-12-2010 08:27:05 PM -06

.remote_site_name => authapp2.mgic.net

14:27:06 : link_name => BAULRZHE.IMS.RSA

.link_status => ENABLED

.link_error =>

.remote_utc_time => 29-12-2010 08:27:05 PM -06

.remote_capture_status => ABORTED

.remote_capture_state =>

.remote_capture_error => ORA-01013: user requested cancel of current operation

.remote_capture_time =>

.remote_used_log_size => 2184

.remote_max_log_size => 102400

.remote_min_log_time => 21-12-2010 04:51:17 AM

.local_propagation_status => ABORTED

.local_propagation_error =>

.local_propagation_num_msgs =>

.remote_propagation_status => ABORTED

.remote_propagation_error =>

.remote_propagation_num_msgs => 368648

.remote_apply_status => ENABLED

.remote_apply_error =>

.remote_apply_time => 28-12-2010 03:53:46 PM

.remote_apply_timeout => TIMEOUT

.remote_heartbeat_received_time => 28-12-2010 09:53:46 PM -06

.local_apply_status => ENABLED

.local_apply_error =>

.local_apply_time => 28-12-2010 03:53:58 PM

.local_apply_timeout => TIMEOUT

.local_heartbeat_received_time => 28-12-2010 09:53:59 PM -06

.

.PL/SQL procedure successfully completed.

.

.SQL> Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production

.With the Partitioning, Data Mining and Real Application Testing options

 

 Done...


Resolution

 Note: All commands described in this solution must be run by rsaadmin account and not by root.

Run the below commands and run the replication status:

#cd utils

./rsautil manage-database ?a exec-sql ?f change-replication-state.sql ?U com.rsa.replication.admin ?A pause

./rsautil manage-database ?a exec-sql ?f change-replication-state.sql ?U com.rsa.replication.admin ?A resume

#./rsautil manage-database -a exec-sql -U com.rsa.replication.admin -f diagnostics/primary_replication_status.sql

 You may still notice that apply status is ABORTED as shown in below output:

bash-3.00$ ./rsautil manage-database -a exec-sql -U com.rsa.replication.admin -f diagnostics/primary_replication_status.sql

Enter Master password: *********

 

 Manage Database ims-2.0.3-build20100922141135

 Copyright (C) 2008 RSA Security Inc. All rights reserved.

 

 %% Running at: authapp1.mgic.net:[r8b4e05i] %%

 

 ===========================================

 %             Execute SQL                %

 ===========================================

 Are you sure you want to execute the SQL file [diagnostics/primary_replication_status.sql] ? (Y/N): y

.

.SQL*Plus: Release 10.2.0.4.0 - Production on Wed Dec 29 14:42:07 2010

.

.Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.

.

.

.Connected to:

.Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production

.With the Partitioning, Data Mining and Real Application Testing options

.

.REPLICATION_REPLICA_APPLY_ERROR

14:42:08 : capture_name => RSA_STREAMS_CAPTURE

.capture_status => ENABLED

.capture_state => CAPTURING CHANGES

.capture_error =>

.capture_time => 29-12-2010 02:42:07 PM

.used_arc_log_size => 2945

.max_arc_log_size => 102400

.min_arc_log_time => 21-12-2010 04:06:01 AM

.local_utc_time => 29-12-2010 08:42:07 PM -06

.remote_site_name => authapp2.mgic.net

14:42:08 : link_name => BAULRZHE.IMS.RSA

.link_status => ENABLED

.link_error =>

.remote_utc_time => 29-12-2010 08:42:07 PM -06

.remote_capture_status => ENABLED

.remote_capture_state => CAPTURING CHANGES

.remote_capture_error =>

.remote_capture_time => 29-12-2010 02:42:02 PM

.remote_used_log_size => 2184

.remote_max_log_size => 102400

.remote_min_log_time => 21-12-2010 04:51:17 AM

.local_propagation_status => ENABLED

.local_propagation_error =>

.local_propagation_num_msgs => 4797

.remote_propagation_status => ENABLED

.remote_propagation_error =>

.remote_propagation_num_msgs => 372094

.remote_apply_status => ABORTED

.remote_apply_error => ORA-26714: User error encountered while applying

.remote_apply_time => 29-12-2010 02:42:00 PM

.remote_apply_timeout =>

.remote_heartbeat_received_time => 28-12-2010 09:53:46 PM -06

.local_apply_status => ENABLED

.local_apply_error =>

.local_apply_time => 29-12-2010 02:42:00 PM

.local_apply_timeout =>

.local_heartbeat_received_time => 29-12-2010 08:42:00 PM -06

.

.PL/SQL procedure successfully completed.

.

.SQL> Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production

.With the Partitioning, Data Mining and Real Application Testing options

 

 Done...


Sp4 installation when fails, it does not restart apply process after restoring to previous version.

 

Run below command to correct the apply status.

 

#cd utils

# . ./rsaenv  ( Notice the two dots )

./rsautil manage-data -a exec-sql -f diagnostics/enable-rep.sql -U com.rsa.replication.admin

 

On Windows:

 

cd utils

rsautil manage-data -a exec-sql -f diagnostics\enable-rep.sql -U com.rsa.replication.admin  

 

Run below command to verify replication status:

 

#./rsautil manage-database -a exec-sql -U com.rsa.replication.admin -f diagnostics/primary_replication_status.sql

 

Compare the output with the output in step3

 

You will notice in the output that all components are ENABLED.

 

 

If you see any errors or TIME OUT displayed in replication status run below reports and send the reports to RSA Technical Support.

 

 

#cd utils

./rsautil manage-data -a exec-sql -f diagnostics/IMS_RepErrorRpt.sql -A error_report.html -U com.rsa.replication.admin -q

./rsautil manage-data -a exec-sql -f diagnostics/IMS_RepInfoRpt.sql -A info_report.html -U com.rsa.replication.admin -q