000032505 - How to extract useful information from RSA Security Analytics core dump files to aid in troubleshooting

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 5Show Document
  • View in full screen mode

Article Content

Article Number000032505
Applies ToRSA Product Set: Security Analytics
RSA Product/Service Type: Core Appliance
RSA Version/Condition: 10.4.x, 10.5.x, 10.6.x
 
IssueOccasionally, a core dump may be created on a Security Analytics appliance. It is useful to know why the core dumps occurred in order to avoid a recurrence.
Unfortunately the core dumps can be very large, and even if compressed can take a long time to transfer from a customer site to development.
This article explains how to extract useful information from a core dump file. This avoids the needs to transfer large core dump files from a customer site to development.
Resolution

Stack-Trace


The script Stack-Trace can be used to collect the process stack information from a running process or process and its corefile.
While debugging issues related to process crash issues or slow performance or process hung it would be helpful if we can get stack trace, iostats and per thread cpu and memory usage from customer environment.
In many cases we observe delay in obtaining the core file from customer end, copying to local share, extracting  debug symbols and obtain process stack .While we are in process of obtaining corefile from customer we can have a quick look at process stack if we can get the Stack-Trace output and use it for initial analysis along with nwtech.
This tool can also be used internally if we would like to get stack trace from a "running process" or "process and corefile" or "executable and its corefile".
Copy the attached Stack-Trace script to device and set executable permissions chmod +x Stack-Trace.
Run:  ./Stack-Trace  or ./Stack-Trace -h for the usage as shown below.
[root@NWAPPLIANCE18184 coredump]# ./Stack-Trace
Usage: Stack-Trace <process-name>
        ex: Stack-Trace NwDecoder
             Collects process info, iostats and process stack trace
       Stack-Trace <process-name> <core-file-path>
        ex: Stack-Trace NwDecoder /var/netwitness/decoder/packetdb/core.12345
             Collects stack trace from corefile
       Stack-Trace <process-full-path> <core-file-path>
        ex: Stack-Trace /home/netwitness/NwDecoder /var/netwitness/decoder/packetdb/core.12345
             Collects stack trace from corefile
NotesPlease redirect the output of command to a text file and send to Support for further analysis.
The Stack-Trace script requires that the package gdb is installed. This can be achieved by typing the following: yum install gdb

Attachments

Outcomes