000029274 - Applying RSA Authentication Manager 8.1 SP1 to a replica first will break replication

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000029274
Applies ToRSA Product Set: SecurID
RSA Product/Service Type: Authentication Manager
RSA Version/Condition: 8.1
IssueThe installation instructions for the Authentication Manager 8.1 SP1 upgrade (Appendix A of the RSA Authentication Manager 8.1 Setup and Configuration Guide, Revision 3) clearly state:
 
Apply SP1 to the primary instance before upgrading the replica instances in your RSA Authentication Manager 8.1 deployment.
 

If  the SP1 upgrade is applied to the replica first, replication will break.  After completing the upgrade by installing SP1 to the primary the replica will be listed as offline in the replication status report in the Operations Console of the primary server.  The following errors will also be in the /opt/rsa/am/server/logs/ReplicaReplication.log on the replica:

 
@@@2014-12-22 10:24:22,495 ERROR [ApplyP2R latestAppliedSweepId: 98915 linesCommittedInNextSweep: 0 nextSweepIdToApply: 98916]
ApplyP2R.executeBatchUpdate(202) | am81r1.vcloud.local,,,,unable to execute batch update when applying primary changes
java.sql.BatchUpdateException: Batch entry 0 insert into rsa_rep.am_attr_categories ( id, label_key, is_editable_ind, domain_object_type ) values
(   E'000000000000000000002001f0020036', E'TOKEN_SOFT_ANDROID_2.x', 'false', E'TOKEN' ) was aborted.  Call getNextException to see the cause.
        at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2598)
        at org.postgresql.core.v3.QueryExecutorImpl$1.handleError(QueryExecutorImpl.java:459)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:407)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2737)
        at com.rsa.replication.ApplyP2R.executeBatchUpdate(ApplyP2R.java:200)
        at com.rsa.replication.ApplyP2R.commitBatches(ApplyP2R.java:175)
        at com.rsa.replication.ApplyThread$1.doWork(ApplyThread.java:64)
        at com.rsa.replication.AutoFileCloser.<init>(AutoFileCloser.java:28)
        at com.rsa.replication.ApplyThread$1.<init>(ApplyThread.java:55)
        at com.rsa.replication.ApplyThread.applyChangesFromFileToDatabase(ApplyThread.java:55)
        at com.rsa.replication.ApplyThread.applyInsideTransaction(ApplyThread.java:49)
        at com.rsa.replication.ApplyThread.apply(ApplyThread.java:36)
        at com.rsa.replication.ApplyP2R.workIfNeccessary(ApplyP2R.java:73)
        at com.rsa.replication.ReplicationRunnable.work(ReplicationRunnable.java:70)
        at com.rsa.replication.util.ServiceCallable.work(ServiceCallable.java:110)
        at com.rsa.replication.util.ServiceCallable.runMainLoopUnsafe(ServiceCallable.java:99)
        at com.rsa.replication.util.ServiceCallable.runMainLoop(ServiceCallable.java:79)
        at com.rsa.replication.util.ServiceCallable.call(ServiceCallable.java:42)
        at com.rsa.replication.util.ServiceCallable.call(ServiceCallable.java:1)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
        at java.lang.Thread.run(Thread.java:680)
@@@2014-12-22 10:24:22,498 FATAL [ApplyP2R latestAppliedSweepId: 98915 linesCommittedInNextSweep: 0 nextSweepIdToApply: 98916]
ServiceCallable.runMainLoop(83) | am81r1.vcloud.local,,,,Unhandled exception during main loop. Shutting down all service threads.
java.lang.RuntimeException: unable to apply primary changes
        at com.rsa.replication.ApplyP2R.wrapAndThrowPossibleDeadlock(ApplyP2R.java:213)
        at com.rsa.replication.ApplyP2R.executeBatchUpdate(ApplyP2R.java:203)
        at com.rsa.replication.ApplyP2R.commitBatches(ApplyP2R.java:175)
        at com.rsa.replication.ApplyThread$1.doWork(ApplyThread.java:64)
        at com.rsa.replication.AutoFileCloser.<init>(AutoFileCloser.java:28)
        at com.rsa.replication.ApplyThread$1.<init>(ApplyThread.java:55)
        at com.rsa.replication.ApplyThread.applyChangesFromFileToDatabase(ApplyThread.java:71)
        at com.rsa.replication.ApplyThread.applyInsideTransaction(ApplyThread.java:50)
        at com.rsa.replication.ApplyThread.apply(ApplyThread.java:36)
        at com.rsa.replication.ApplyP2R.workIfNeccessary(ApplyP2R.java:73)
        at com.rsa.replication.ReplicationRunnable.work(ReplicationRunnable.java:73)
        at com.rsa.replication.util.ServiceCallable.work(ServiceCallable.java:110)
        at com.rsa.replication.util.ServiceCallable.runMainLoopUnsafe(ServiceCallable.java:99)
        at com.rsa.replication.util.ServiceCallable.runMainLoop(ServiceCallable.java:80)
        at com.rsa.replication.util.ServiceCallable.call(ServiceCallable.java:42)
        at com.rsa.replication.util.ServiceCallable.call(ServiceCallable.java:1)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:139)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
        at java.lang.Thread.run(Thread.java:680)
Caused by: java.sql.BatchUpdateException: Batch entry 0 insert into rsa_rep.am_attr_categories ( id, label_key, is_editable_ind, domain_object_type ) values
(   E'000000000000000000002001f0020036', E'TOKEN_SOFT_ANDROID_2.x', 'false', E'TOKEN' ) was aborted.  Call getNextException to see the cause.
        at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2598)
        at org.postgresql.core.v3.QueryExecutorImpl$1.handleError(QueryExecutorImpl.java:459)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:407)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2737)
        at com.rsa.replication.ApplyP2R.executeBatchUpdate(ApplyP2R.java:200)
        at com.rsa.replication.ApplyP2R.commitBatches(ApplyP2R.java:175)
        at com.rsa.replication.ApplyThread$1.doWork(ApplyThread.java:64)
        at com.rsa.replication.AutoFileCloser.<init>(AutoFileCloser.java:28)
        at com.rsa.replication.ApplyThread$1.<init>(ApplyThread.java:55)
        at com.rsa.replication.ApplyThread.applyChangesFromFileToDatabase(ApplyThread.java:55)
        at com.rsa.replication.ApplyThread.applyInsideTransaction(ApplyThread.java:49)
        at com.rsa.replication.ApplyThread.apply(ApplyThread.java:36)
        at com.rsa.replication.ApplyP2R.workIfNeccessary(ApplyP2R.java:73)
        at com.rsa.replication.ReplicationRunnable.work(ReplicationRunnable.java:70)
        at com.rsa.replication.util.ServiceCallable.work(ServiceCallable.java:110)
        at com.rsa.replication.util.ServiceCallable.runMainLoopUnsafe(ServiceCallable.java:99)
        at com.rsa.replication.util.ServiceCallable.runMainLoop(ServiceCallable.java:79)
        at com.rsa.replication.util.ServiceCallable.call(ServiceCallable.java:42)
        at com.rsa.replication.util.ServiceCallable.call(ServiceCallable.java:1)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
        ... 1 more

In /opt/rsa/am/server/logs/ReplicaReplicationWrapper.log file, the following errors display:
ERROR  | wrapper  | main    | 2014/12/22 10:24:22 | JVM exited unexpectedly.
STATUS | wrapper  | main    | 2014/12/22 10:24:26 | Launching a JVM...
INFO   | jvm 2    | main    | 2014/12/22 10:24:27 | Wrapper (Version 3.2.3rsa4) http://wrapper.tanukisoftware.org
INFO   | jvm 2    | main    | 2014/12/22 10:24:27 |   Copyright 1999-2006 Tanuki Software, Inc.  All Rights Reserved.
INFO   | jvm 2    | main    | 2014/12/22 10:24:27 |
INFO   | jvm 2    | main    | 2014/12/22 10:24:29 | The most recent invocation of this service was not shutdown cleanly
WARN   | wrapper  | main    | 2014/12/22 10:24:30 | JVM exited unexpectedly while stopping the application.
STATUS | wrapper  | main    | 2014/12/22 10:24:30 | on_exit trigger matched.  Restarting the JVM.  (Exit code: 1)
STATUS | wrapper  | main    | 2014/12/22 10:24:34 | Launching a JVM...
INFO   | jvm 3    | main    | 2014/12/22 10:24:35 | Wrapper (Version 3.2.3rsa4) http://wrapper.tanukisoftware.org
INFO   | jvm 3    | main    | 2014/12/22 10:24:35 |   Copyright 1999-2006 Tanuki Software, Inc.  All Rights Reserved.
CauseReplication breaks if Authentication Manager 8.1 SP1 is installed on a replica before installing it on the primary, as instructed in the RSA Authentication Manager 8.1 Setup and Configuration Guide.
ResolutionThe replication system can be forced to re-synchronize by altering a field in the primary server's database. Please contact RSA support for assistance, as this is not a published procedure.
WorkaroundIn lieu of calling RSA customer support, resolve the issue by deleting the replica Authentication Manager server from the Operations Console of the primary Authentication Manager server  (Deployment Configuration > Instances > Manage Existingthen redeploying it. After redeploying the replica, SP1 will need to be applied to the replica after it is attached to the primary.
 

Attachments

    Outcomes