000029517 - Custom feed is not being applied to all meta data in RSA Security Analytics

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 4Show Document
  • View in full screen mode

Article Content

Article Number000029517
Applies ToRSA Product Set: Security Analytics
RSA Product/Service Type: Decoder, Log Decoder, Security Analytics UI
RSA Version/Condition: All Versions
Platform: CentOS
O/S Version: EL6
IssueAfter creating a custom feed, for example to tag meta against a set of Fully Qualified Domain Names (FQDN), only some of the entries are tagged by the feed whereas some are not.
In this example, the CSV file being used is shown in the screenshot below.
 
CSV example


The XML file for the feed in this example is shown below.
<?xml version="1.0" encoding="UTF-8"?>
<FDF>
<FlatFileFeed separator="," comment="#" path="CompanyServers.csv" name="CompanyServers">
  <MetaCallback name="device" valuetype="Text">
   <Meta name="device.host">
            </Meta>
  </MetaCallback>
  <LanguageKeys>
   <LanguageKey name="companyserversfqdn" valuetype="Text"/>  
  </LanguageKeys>
  <Fields>
   <Field type="index" index="1"/>
   <Field type="value" index="2" key="companyserversfqdn"/>
  </Fields>
</FlatFileFeed>
</FDF>
CauseBy default, custom feeds are case-sensitive. Therefore, if meta is generated with different casing from that which is in the CSV file, the custom feed will not tag the meta properly.
In the example above, the CSV file has an entry for the domain1.example.com FQDN. If meta is generated for Domain1.EXAMPLE.COM then it will not be tagged by the custom feed.
ResolutionThe basic feed setup in the UI is great for putting together a quick feed that does not have any special requirements with respect to the data it includes.
 
However, If you want to do anything special, like multi-indexed feeds, SRC/DST meta key feeds, or in this case making it case insensitive, you have to manually write the .xml file for the feed.
 
Prior to Security Analytics, in Netwitness feeds were written by hand as there was no UI mechanism to create them. 
 
The *.feed files on the Log Decoder in /etc/netwitness/ng/feeds are compiled from the .csv and the .xml file (either automatically created from using the UI, or manually created and uploaded with the CSV in the advanced section of the custom feed wizard in the UI).
 
Additionally, once created, custom feed files are also located on the SA Server in /var/lib/netwitness/uax/temp in .zip format (containing the .csv file and an expanded xml file).
 
If you want to build a custom XML, begin by grabbing a copy of the .xml from that zip file to modify.
 
If you open the .xml file in an editor like vi, the lines will just appear jumbled together.
 
It is recommended to use a separate editor, something that supports XML formatting, to manage the XML file.
 
A copy of it will need to be pulled off the system anyways as, once modified, it will be linked back into the the custom feed definition.
 
Optional: Notepad++ with the XML tools plugin may be an editor to use.
Optional: In Notepad++, paste the contents into a new file and then from the plugins menu, in the XML tools submenu, select "Pretty Print (XML Only - with line breaks)" (this will put it in a more readable format).

To make the custom feed case-insensitive, the ignorecase boolean value must be set to true within the MetaCallback tag in the XML file.  This is shown in the example below:
<?xml version="1.0" encoding="UTF-8"?>
<FDF>
<FlatFileFeed separator="," comment="#" path="CompanyServers.csv" name="CompanyServers">
  <MetaCallback name="device" valuetype="Text" ignorecase="true">
   <Meta name="device.host"></Meta>
  </MetaCallback>
  <LanguageKeys>
   <LanguageKey name="companyserversfqdn" valuetype="Text"/>
  </LanguageKeys>
  <Fields>
   <Field type="index" index="1"/>
   <Field type="value" index="2" key="companyserversfqdn"/>
  </Fields>
</FlatFileFeed>
</FDF>

Once the changes have been made, go to Live -> Feeds, edit the custom feed and add the modified .xml file to the Advanced section on the first wizard window.
 
Continue stepping through (it will look a bit different since a customized .xml was uploaded) and save.
 
The feed should now work with incoming data no matter what the case.

 

Attachments

    Outcomes