Leonard Chvilicek

RSA Netwitness Suite Log Parser 2.3.99

Blog Post created by Leonard Chvilicek Employee on May 21, 2018

Overview

This version will now parse over 1,400 events from the devices, however the parser does not parse audit events that are generated in the "Administration-->Security" user interface.  Those events are handled by the Global Audit, Global Notification settings and parsed by the CEF parser.  However, if you made modifications to the "Security" settings on the individual device, that event will be parsed by this parser.

This version was developed and tested on 10.6.2.0 using available log samples from 10.4.x thru 10.6.2.0.

 

Improvements

New Headers have been added to accommodate the log format change in 10.5.1 and above.

Logs from the Virtual Log Collector are now parsed, particularly Windows Collection Errors.

Error/Failure Logs are consolidated under the Event Category Name of "System.Errors"

Puppet Logs are parsed

Collectd Logs are parsed

Added "maxValues" kb 00031300 modification

Custom Index reduction in size and maxValues adjusted accordingly

Overall cleanup of some variable/index clutter

Improved accuracy for parsing of Query and Queue Times

Duration added for Query Times, they are now converted to seconds under the "duration.time" metakey

 

Contents

This package includes:

   Custom Log parser

   Custom Index for Concentrator*

   Custom Table Map*
   Event Categories Spreadsheet

  

*I have revised the custom index and table map to reflect the new changes in the default settings for 10.6.2.  If you are using a prior version to 10.6, you may need to add some additional index keys to the custom index.

 

Parser Content

Content, such as reports and dashboards, written by me for this parser will be published separately and links will be added here.  Currently content for Index operations, queries, cancelled queries, system errors, configuration changes, security changes, service restarts, and content updates for feeds/parsers are being tested on an enterprise system at the time of this writing.  These will start appearing in the next few days.

 

Report:  ValueMax Has Been Reached 

 

Installation

Log Decoder

Remove the prior version of the parser

  1. SSH into each log decoder as "root" that has the prior version.
  2. Remove the old parser directory
    rm -r /etc/netwitness/ng/envision/etc/devices/rsasecurityanalytics/
    You should see the prompts like below:
    [root@logdecoder60 SA_Logs]# rm -r /etc/netwitness/ng/envision/etc/devices/rsasecurityanalytics/
    rm: descend into directory `/etc/netwitness/ng/envision/etc/devices/rsasecurityanalytics'? y
    rm: remove regular file `/etc/netwitness/ng/envision/etc/devices/rsasecurityanalytics/rsasecurityanalytics.ini'? y
    rm: remove regular file `/etc/netwitness/ng/envision/etc/devices/rsasecurityanalytics/rsasecurityanalyticsmsg.xml'? y
    rm: remove directory `/etc/netwitness/ng/envision/etc/devices/rsasecurityanalytics'? y

Download and unzip parser

  1. Download the parser file "rsasecurityanalytics_2.3.99.zip" from the bottom of this page.
  2. Unzip the file using Winzip, or 7zip.
    The unzipped parser file name will be "rsasecurityanalytics.envision"

Upload the parser on the Log Decoder

  1. Login to the Web Interface as "admin" or user who is a member of the "Administrators" Role.
  2. Choose "Administration-->Services" from the navigation menu in the upper left corner of the screen.
  3. Locate the Log Decoder and click on the gear icon, located at the far right of the screen.
  4. Hover over "View", then click "Config".
  5. Click on the "Parsers" Tab.
  6. Click on the "Upload" icon in the upper left portion of the window.
  7. Click on the "+" in the upper left of the "Upload Parsers" dialog box.
  8. Navigate to the folder where the "rsasecurityanalytics.envision" is located and select it.  Click "Open"
  9. Click on "Upload"
  10. Click on the "X" in the upper right corner of the dialog box or click "Cancel"

Remove prior version custom table map entries

  1. On the same screen as above, Click on the "Files" Tab
  2. On the left side of the screen click on the dropdown and select "table-map-custom.xml".
  3. Locate the section related to the custom table entries for the log parser typically labelled
    RSA Security Analytics Log Parser Revision 2.1.63 xx/xx/xx
  4. Remove that section.
  5. Replace with new table map entries from the table-map-custom.xml file.
  6. Click "Apply"

Load the new log parser and custom table map.

  1. On the same screen as above, click on "Config" just above the "App Rules" Tab.
  2. Click on "System"
  3. Click on "Stop Capture" at the top left of the screen.
  4. Wait for capture to stop.
  5. Click on "Shutdown Service" at the top center of the screen.
  6. On the "Confirm Shutdown" dialog, type "RSA Security Analytics Parser update"
  7. Click "OK"

Concentrator

Update The Concentrator Custom Index

  1. Login to the Web Interface as "admin" or user who is a member of the "Administrators" Role.
  2. Choose "Administration-->Services" from the navigation menu in the upper left corner of the screen.
  3. Locate the Concentrator and click on the gear icon, located at the far right of the screen.
  4. Hover over "View", then click "Config".
  5. Click on the "Files" Tab
  6. On the left side of the screen click on the dropdown and select "index-concentrator-custom.xml".
  7. Locate the section related to the custom table entries for the log parser typically labelled
    RSA Security Analytics Log Parser Revision 2.1.63 xx/xx/xx
  8. Remove that section.
  9. Replace with new custom index entries from the index-concentrator-custom.xml file.
  10. Click "Apply"

Load The New Custom Index.

  1. On the same screen as above, click on "Config" just above the "Correlation Rules" Tab.
  2. Click on "System"
  3. Click on "Stop Aggregation" at the top left of the screen.
  4. Wait for aggregation to stop.
  5. Click on "Shutdown Service" at the top center of the screen.
  6. On the "Confirm Shutdown" dialog, type "RSA Security Analytics Parser update"
  7. Click "OK"

ALL Appliances

Configure Rsyslog to Forward Logs

  1. SSH into each NetWitness Appliance.
  2. Modify the /etc/rsyslog.conf file.  
    vi /etc/rsyslog.conf
  3. Press the letter "i" or the "Insert" key.  You should see "-- INSERT --" at the bottom left of your screen.
  4. Scroll to the bottom of the file and look for the following line:
    #*.* @@remote-host:514
  5. Remove the "#" and change "remote-host" to the destination Log Decoder or Virtual Log Collector (VLC).
    *.* @@<Log Decoder or VLC IP Address Here>:514
  6. Press the  "ESC" key
  7. You should see a colon ":" in the lower left of the screen.
  8. Save the file by typing ":wq"
    :wq
  9. Restart the Rsyslog service.
    service rsyslog restart
  10. Rsyslog is now forwarding logs to the Log Decoder or VLC.

Outcomes