000030142 - How to monitor for core files on an RSA Security Analytics core appliance

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 4Show Document
  • View in full screen mode

Article Content

Article Number000030142
Applies ToRSA Product Set: Security Analytics
RSA Product/Service Type: Decoder, Log Decoder, Concentrator, Broker, Hybrid, All-in-One
RSA Version/Condition: 10.3.x, 10.4.x
Platform: CentOS
IssueWhen a core appliance process (i.e. nwdecoder, nwconcentrator, etc.) crashes, a core dump file is generated containing the process's memory at the time of termination.
It is important to identify when these files are present as they generally take up a large amount of disk space which can result in issues with the functionality of the appliance.
Core dump files can also be useful in performing a root cause analysis to determine what caused the process to crash.
ResolutionAttached to this article is a core file detection script entitled nwcorecount.sh, which is intended to be used as an hourly or daily cron job on Security Analytics core appliances.
When executed, the script will scan the /var/netwitness subdirectories in order to locate any core dump files that are present.
If such files are found, an entry will be logged in the /var/log/messages file stating how many files were found and directing the user to the /var/log/nwcorecount.log file where additional information is located.
Below is sample output from a Packet Decoder appliance on which the script was executed.
[root@PDecoder ~]# ./nwcorecount.sh
[root@PDecoder ~]# tail -1 /var/log/messages
Apr 26 12:35:47 PDecoder nwcorecount.sh: 2 core file(s) found for NetWitness services.  See /var/log/nwcorecount.log for more information.
[root@PDecoder ~]# cat /var/log/nwcorecount.log
Sun Apr 26 12:35:47 UTC 2015
2 core file(s) found.
/var/netwitness/decoder/packetdb/core.5678
/var/netwitness/decoder/packetdb/core.1234
[root@PDecoder ~]#

To utilize the script, simply transfer it as the root user to either the /etc/cron.hourly or /etc/cron.daily directory on a core appliance and--from the directory in which the script was placed--issue the command chmod +x nwcorecount.sh to mark it as an executable.

Attachments

Outcomes