000036059 - RSA RADIUS Server Operations Console service for RSA Authentication Manager 8.2 and higher fails to start after upgrade

Document created by RSA Customer Support Employee on Feb 22, 2018Last modified by RSA Customer Support Employee on Jun 25, 2018
Version 5Show Document
  • View in full screen mode

Article Content

Article Number000036059
Applies ToRSA Product Set: SecurID  
RSA Product/Service Type: Authentication Manager
RSA Version/Condition: 8.2, 8.3
IssueThe RSA RADIUS Server Operations Console service fails to start after an upgrade.

A search of the /opt/rsa/am/server/logs/radiusoc.log shows:   

weblogic.application.ModuleException: java.lang.OutOfMemoryError: Java heap space

CauseA WebLogic Diagnostic Framework file has grown in size exceeding the limits defined in the configuration that Authentication Manager uses to run WebLogic.

The processing of this large file size causes the aforementioned OutOfMemory Error.  This has mostly been seen when patching, where the file has grown large and the patch process performs a stop and start of services.  It can happen any time a stop and start is done.  When the file is too large in size to impact the start up process varies. 
  1. Remove the excessively large file WLS_DIAGNOSTICS000000.DAT
  2. Login to the RSA Authentication Manager server via SSH as rsaadmin. 
  3. Navigate to /opt/rsa/am/
  4. Run a find command as follows:

find . -iname WLS_DIAG*.DAT -exec ls -lh {} \+

  1. Output will look similar to the following:

-rw------- 1 rsaadmin rsaadmin 86M Mar  7 16:00 ./server/servers/AdminServer/data/store/diagnostics/WLS_DIAGNOSTICS000000.DAT
-rw------- 1 rsaadmin rsaadmin 86M Mar  7 16:00 ./server/servers/biztier/data/store/diagnostics/WLS_DIAGNOSTICS000000.DAT
-rw------- 1 rsaadmin rsaadmin 86M Mar  7 15:59 ./server/servers/console/data/store/diagnostics/WLS_DIAGNOSTICS000000.DAT
-rw------- 1 rsaadmin rsaadmin 1.4G Mar  7 15:59 ./server/servers/radiusoc/data/store/diagnostics/WLS_DIAGNOSTICS000000.DAT

  1. Delete the file shown above, which is large.

cd server/servers/radiusoc/data/store/diagnostics/

  1. Navigate back to /opt/rsa/am/server:

cd /opt/rsa/am/server

  1. Start all services:

./rsaserv stop all
./rsaserv start all

WorkaroundAt this time the cause of the file size growth is unknown.  It is a benign file for the way RSA uses WebLogic.  Engineering is investigating the cause and remedial steps to constrain the growth of the file. 

This issue has been addressed in AM-31774 (Huge file WLS_DIAGNOSTICSxxxxxx.DAT preventing 3 services from starting).  The fix is included in  RSA Authentication Manager 8.3 Patch 1 or later (requires RSA Link registration).