on Sep 12, 2017The custom feed capability is implemented using the Custom Feed Wizard in RSA NetWitness Platform, allowing you to quickly populate Decoders with custom and identity feeds.
Custom Feed Creation
You can use the Live > Custom Feeds > Setup Feed > Configure a Custom Feed wizard to create and deploy Decoder feeds based on deterministic logic that offers the meta keys specific to the selected Decoders and Log Decoders. Although the wizard guides you through the process to create both on-demand and recurring feeds, you should understand the form and content of a feed file when you create a feed.
Feed file names in RSA NetWitness Platform are in the form <filename>.feed. To create a feed, NetWitness Platform requires a feed data file in .csv or .xml (for STIX) format and a feed definition file in .xml format, which describes the structure of a feed data file. The Configure a Custom Feed wizard can create the feed definition file based on a feed data file, or based on a feed data file and the corresponding feed definition file.
The files that you use to create an on-demand feed must be stored on your local file system. The files used to create a recurring feed must be stored at an accessible URL, whence NetWitness Platform can fetch the most current version of the file for each recurrence. After a NetWitness Platform feed is created, you can download the feed to your local file system, edit the feed files, and edit the NetWitness Platform feed to use the updated feed files.
Sample Feed Definition File
This is an example of a feed definition file named dynamic_dns.xml, which NetWitness Platform creates based on your entries in the Feed wizards. It defines the structure of the feed data file named dynamic_dns.csv.
Note: The feed file path should be .csv regardless of the Feed Type (Default or STIX).
Sample Feed Definition File
| <?xml version="1.0" encoding="utf-8"?> |
| <FDF xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="feed-definitions.xsd"> |
| <FlatFileFeed name="Dynamic DNS Domain Feed" |
| path="dynamic_dns.csv" |
| separator="," |
| comment="#" |
| version="1"> |
| <MetaCallback |
| name="alias.host" |
| valuetype="Text" |
| apptype="0" |
| truncdomain="true"/> |
| <LanguageKeys> |
| <LanguageKey name="threat.source" valuetype="Text" /> |
| <LanguageKey name="threat.category" valuetype="Text" /> |
| <LanguageKey name="threat.desc" valuetype="Text" /> |
| </LanguageKeys> |
| <Fields> |
| <Field index="1" type="index" key="alias.host" /> |
| <Field index="4" type="value" key="threat.desc" /> |
| <Field index="2" type="value" key="threat.source" /> |
| <Field index="3" type="value" key="threat.category" /> |
| </Fields> |
| </FlatFileFeed> |
| </FDF> |
Feed Definition Equivalents for Custom Feed Wizard Parameters
The NetWitness Platform Feeds wizard provide options to define the structure of the data feed file. These correspond directly to attributes in the feed definition (.xml) file.
| NetWitness Platform Parameter | Feed Definition File Equivalent |
|---|---|
| Define Feed tab | |
| Feed Type | Select: Default - to define a feed based on a .csv formatted feed data file. STIX - to define a feed based on STIX formatted .xml file. |
| Feed Task Type | Select: Adhoc - to create an on-demand feed. Recurring - to create a feed that recurs automatically. |
| Name | Enter a custom feed name in the feed data file that corresponds to the flatfeedfile name attribute in the feed definition file; for example, Dynamic DNS Test Feed. |
| File/ Browse | Enter a name of the feed data file that corresponds to the flatfeedfile path attribute in the feed definition file; for example, dynamic_dns.csv. |
| (STIX, Recurring) Trust All Certificate | Select Trust All Certificate, if you do not want to validate the REST server certificate. This option is enabled by default (checked). |
| (STIX, Recurring) Certificate/Browse | For client authentication with the REST URL, in the Certificate field, click Browse and select the self signed certificate. The supported certificate formats are .cer, .crt with Base64 & DER encoded files. |
| Define Feed tab - Advanced Options | |
| XML Feed File | Enter a name of the feed definition file, for example, dynamic_dns.xml. |
| Separator | The separator character used to separate attributes in the feed data file. It corresponds to the flatfeedfile separator in the feed definition file; for example, a comma. |
| Comment | The character used to identify a comment in the feed data file. It corresponds to the flatfeedfile comment attribute in the feed definition file; for example, #. |
| Remove STIX data older than | The number of days for which the STIX packages downloaded from TAXII server have to be stored. The STIX packages older than the specified number of days are deleted automatically. The default value is 180 days, which is also the maximum. |
| Select Services tab | Select the services to which you want to send the data feed. |
| (Define Columns tab, Define Index) Type | The type of lookup value in the index position of the feed data file. |
| (Define Columns tab, Define Index) CIDR | Specifies that the IP value in the lookup position is in CIDR format. The CIDR attribute sets the IP address format in the field to Classless Inter-Domain Routing (CIDR) notation. |
| (Define Columns tab, Define Index) Service Type | For a Non IP index, the integer service type to filter meta lookups. It corresponds to MetaCallback apptype attribute in the feed definition file. A value of 0 indicates no filtering by service type. |
| (Define Columns tab, Define Index) Truncate Domain | For a Non IP index, for meta values that contain domain names (for example, hostnames), the system can strip off the host specific element in the data. Truncate Domain corresponds to the MetaCallback truncdomain attribute. If the value is www.example.com, it is truncated to example.com. A value of False selects no truncation, and True selects truncation. |
| (Define Columns tab, Define Index) Callback Keys | For a Non IP index, the available meta keys to match on instead of ip.src/ip.dst (the defaults for IP index type) are selectable from the drop-down list. The Callback Key corresponds to the MetaCallback name attribute, and the index column of the csv file must contain data that can match the chosen meta key. For example, if the username meta key is chosen, the index column of the csv file needs to be populated with users to be matched. |
| (Define Columns tab, Define Index) Index Column | Identifies the column in the feed data file that provides the lookup value for the row. Each position in each row of the feed data file is identified by a Field index attribute in the feed definition file. A field with an index of 1 is the first entry in a row, the second field has an index of 2, the third field has an index of 3, and so on. You can select multiple index columns, if the Feed Type is STIX and Index Type is Non IP. When you select multiple index columns the values from all the selected columns are merged in the first index column that you selected. |
| (DEFINE VALUES) Key | The name of the LanguageKey, as defined in the feed definition file, for which meta is created from this row of the feed data file. It corresponds to the Field key attribute in the feed definition file. A key applies only to a field whose type is set to value. In the feed definition file, there is a list of LanguageKeys from index.xml, or a summary name if Source Name and Destination Name are used. For example, reputation is a summary name for reputation.src and reputation.dst). This value is referenced by the Field key attribute. |
Next steps
