RSA Replication Service fails to start on RSA Authentication Manager 8.x primary
Originally Published: 2019-11-29
Last Modified: 2024-11-13
Article Number
Applies To
RSA Product/Service Type: Authentication Manager
RSA Version/Condition: 8.x
Issue
- The RSA Replication (Primary) Service on a primary instance shows a status of SHUTDOWN, as shown:
rsaadmin@primary:/opt/rsa/am/server> ./rsaserv status
RSA Database Server [RUNNING]
RSA Administration Server with Operations Console [RUNNING]
RSA RADIUS Server Operations Console [RUNNING]
RSA Runtime Server [RUNNING]
RSA RADIUS Server [RUNNING]
RSA Console Server [RUNNING]
RSA Replication (Primary) [SHUTDOWN]
- When trying to view the Replication Status Report from the Operations Console of the primary, the following error displays:
The primary instance cannot continue replication. Replication status will not be displayed. If the replication service is stopped, please start it. Otherwise, call RSA Customer Support.
- When restarting the service using the command ./rsaserv start primary_replication, the service shows as RUNNING, then fails again after few seconds.
- There is no issue with disk space.
- System date and time are correct.
- The /opt/rsa/am/server/logs/PrimaryReplication.log shows the following errors:
@@@2019-11-28 04:40:52,967 SYSTEM [WrapperSimpleAppMain ] Service.start(88) | XXXX,,,,Starting the service. @@@2019-11-28 04:41:00,157 FATAL [WriteP2R nextSweepId: 22203619 mode: TMR minPrimaryChangeId: 257689041 primaryChangeIdRange: 221161 primaryChangeIdRangeHWM: 9223372036854775807] ServiceCallable.runMainLoop(83) | XXXX,,,,Unhandled exception during main loop. Shutting down all service threads. java.lang.RuntimeException: unable to obtain primary changes from database at com.rsa.replication.WriteP2R.createSweepFile(WriteP2R.java:209) at com.rsa.replication.WriteP2R.sweepFromInsideTransaction(WriteP2R.java:191) at com.rsa.replication.WriteP2R.sweep(WriteP2R.java:239) at com.rsa.replication.WriteP2R.workIfNeccessary(WriteP2R.java:166) at com.rsa.replication.ReplicationRunnable.work(ReplicationRunnable.java:77) at com.rsa.replication.util.ServiceCallable.work(ServiceCallable.java:110) at com.rsa.replication.util.ServiceCallable.runMainLoopUnsafe(ServiceCallable.java:99) at com.rsa.replication.util.ServiceCallable.runMainLoop(ServiceCallable.java:79) at com.rsa.replication.util.ServiceCallable.call(ServiceCallable.java:42) at com.rsa.replication.util.ServiceCallable.call(ServiceCallable.java:1) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) at java.lang.Thread.run(Thread.java:745) Caused by: org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL []; Ran out of memory retrieving query results.; nested exception is org.postgresql.util.PSQLException: Ran out of memory retrieving query results. at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:257) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:605) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:639) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:664) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:684) at com.rsa.replication.util.WrappedJdbcTemplate.query(WrappedJdbcTemplate.java:62) at com.rsa.replication.WriteP2R$2.sweepUsingAppropriateQuery(WriteP2R.java:106) at com.rsa.replication.WriteP2R.createSweepFile(WriteP2R.java:207) ... 13 more Caused by: org.postgresql.util.PSQLException: Ran out of memory retrieving query results. at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1817) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:273) at com.rsa.replication.util.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:64) at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:646) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:589) ... 19 more Caused by: java.lang.OutOfMemoryError: Java heap space at org.postgresql.core.PGStream.ReceiveTupleV3(PGStream.java:364) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1814) ... 26 more @@@2019-11-28 04:41:00,199 SYSTEM [Shutdown Hook ] Service.shutdownVm(198) | XXXX,,,,Stopped the service.
Cause
Resolution
- Before moving forward with the steps below, please login to the Operations Console on your primary and take a backup (Maintenance > Backup > Backup Now).
- Launch an SSH client, such as PuTTY.
- Login to the primary Authentication Manager server as rsaadmin and enter the operating system password.
- Navigate to /opt/rsa/am/server/wrapper
cd /opt/rsa/am/server/wrapper
- Take a backup of the PrimaryReplicationWrapper.conf as below:
cp PrimaryReplicationWrapper.conf PrimaryReplicationWrapper.conf.ORIG
- Edit the PrimaryReplicationWrapper.conf using:
vim PrimaryReplicationWrapper.conf
- Update the following parameters from
# Server-specific JVM params wrapper.java.additional.18=-Xms20m wrapper.java.additional.19=-Xmx128m
to what is shown in bold:
# Server-specific JVM params wrapper.java.additional.18=-Xms128m wrapper.java.additional.19=-Xmx256m
- Save and exit the file using :wq!
- Restart the primary_replication service:
/opt/rsa/am/server/rsaserv restart primary_replication
Related Articles
How to failover a Citrix NetScaler to an Authentication Manager RADIUS Replica Server 306Number of Views How to successfully authenticate users via RSA ACE/Server on UNIX when using Network Address Translation (NAT) 51Number of Views Authentication Manager Primary Replication Service Fails to Start with "Could Not Acquire Lock" 1.36KNumber of Views "java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space" errors occur frequently in versions 6.x of RSA I… 76Number of Views "The features parameter is not in a valid JSON format" error opening escalation workflow on RSA Governance & Lifecycle 23Number of Views
Trending Articles
RSA Authentication Manager 8.9 Patches and Hotfixes Readme RSA MFA Agent 2.5 for Microsoft Windows Installation and Administration Guide RSA-2026-24: RSA Authentication Manager Security Update for Third-Party Component Vulnerabilities IDR SSO - Step 3: Deploy the Identity Router RSA Authentication Manager 8.9 Setup and Configuration Guide
Don't see what you're looking for?