When troubleshooting some Authentication Manager problems such as increasing swap usage or console unavailable 503 errors / [STUCK] Thread errors, RSA Engineering may ask for the java heap dump output for one or more processes that consume the most Memory.
This Knowledge Base, KB article explains how to Identify those top processes and collect Java Heap Dump information for them.
Run Linux top command,
Note the top three java process IDs or PIDs, which will be input into.
jcmd <PID> GC.heap_dump Which will output the Java heap dump information for that Process ID to a file,
Send the 3 output files to RSA Support for analysis by Engineering.
Engineering may ask you to repeat this process at a later date, but before any restart of AM services or reboot, to monitor the growth of the memory allocations, e.g., problems monitoring increasing swap usage.
How to get Java heap dumps on the Process IDs, pids for the highest java memory using processes for a problem like increasing Swap usage or Console unavailable – before you restart AM services – are as follows.
Short answer – Run Linux top command, not top three java process IDs then run 3 jcmd 25460 GC.heap_dump commands against those PIDs with output to files, send files to RSA.
Details: You identify these processes with the Linux top command
Should get files in /tmp with a message showing PID and <pid>: Heap dump file created Example screenshot for PID 25480 from ‘top’ Image description <screenshot - Java_heap_top_PID.png>
======errors====== I tested to try to get a 0-byte output file in /tmp, but was not exactly successful in doing that, saw other errors.
If I use a nonexistent or fake PID, I get an error and not output file. /opt/rsa/am/appserver/jdk/bin/jcmd 3938 GC.heap_dump /tmp/java2.hprof 3938: java.io.IOException: No such process
If I use an empty, 0-byte /opt/rsa/am/appserver/jdk/bin/jcmd no output file is created.
If I try modifying the GC.heap_dump option after jcmd, no output file and error. /opt/rsa/am/appserver/jdk/bin/jcmd 26692 GD.heap_dump /tmp/java6.hprof 26692: java.lang.IllegalArgumentException: Unknown diagnostic command
/opt/rsa/am/appserver/jdk/bin/jcmd 26692 GC.heap_dump /tmp/java2.hprof 26692: File exists. Will not overwrite.
If I use a PID from a process that is not java, like Adminwrapper, I get a different error. com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file: target process not responding or HotSpot VM not loaded ====end====errors=====