Skip navigation
All Places > Products > RSA NetWitness Platform > Blog > 2016 > July > 01
William Hart

Email Parsing Options

Posted by William Hart Employee Jul 1, 2016

We have got several requests to generate additional parser updates to some of the parsers we distribute in Live. One approach we have taken to make these optional changes available for customers is to allow for an external configuration file to be read by the original parser. This provides for some customization depending on environment requirements while eliminating the need for customers to be able to write a parser. The reason for them being optional is some configuration changes may generate more meta requiring further storage or may require additional parsing resources or not be appropriate to an environment.


As an example the default email parser, MAIL_Lua, reads email messages regardless of transport protocol (e.g. SMTP, IMAP, POP3) and registers all email addresses into the email meta key. In the options file a user can enable to register the email sender to email.src and the recipient to email.dst instead of registering them all to email. There are additional options available in the attached MAIL_lua_options.lua file that can be enabled/disabled as well.


To have these options take effect the following steps are required.


1) Upload the MAIL_lua_options.lua file into the /etc/netwitness/ng/parsers folder on the appropriate decoder where the MAIL_Lua parser is applied.


2) To enable the source and destination email change mentioned above modify the word false to true on line 24. This is the last line before the end of the function named registerEmailSrcDst


3) Validate that the default email source and destination meta keys are included in the appropriate concentrator default index file (e.g. index-concentrator.xml) located in /etc/netwitness/ng on the concentrator.  This file can be viewed by command line after logging in using secure shell or in administration section of the user interface at Administration > Services > <concentrator name> > config > Files tab. The lines that should be in there and that will be populated by this change are:


<key description="Source E-mail Address" level="IndexValues" name="email.src" format="Text" valueMax="2500000" />

<key description="Destination E-mail Address" level="IndexValues" name="email.dst" format="Text" valueMax="2500000" />


4) For this to take effect, a decoder service restart is required. This will cause a service interruption so recommend making this change during a maintenance window.


I welcome any suggestions on importance of these types of options for this scenario as well as others. As well if it would be better to have these options available in the Security Analytics user interface.


Note: I am just a conduit for this information and have to give credit for the creation of these parser options to RSA Content Engineers.

Are you looking for a way to trigger those PCAP downloads so they automatically open in a third party tool? There is a way to do this in Security Analytics 10.4 and above. It does require enabling some settings that may not be enabled by default depending on which version of Security Analytics you are running.


To make your PCAP extractions more efficient do the following steps.


1) Make sure the Download Completed PCAPs setting is enabled. This is available in the Security Analytics interface through the Investigate > Navigate > Settings widget as shown below. The download will still be tracked in the download job queue on the SA server but after completion of the download it will save it to your client machine in your browsers designated download folder.





2) Optionally setup file associations on your operating system so that files with a .pcap extension open in your tool of choice, say Wireshark for example.


Note: Although you can configure this method for downloading files (other then PCAPs) I do not suggest it unless the system you are running your browser on is a machine you are allowed to download and execute malware on. By that I mean the machine is on a segmented network or is a sandboxed virtual machine or some other endpoint software is in place to limit the effect of malware. The reason I bring up this warning is that typically the files being downloaded from Security Analytics are ones suspected of being malware or related to malware and if they are automatically opened by their native application you could end up infecting your own system.

Filter Blog

By date: By tag: