Feeds and parsers are responsible for analyzing the packets and logs when captured or imported in a Decoder or Log Decoder. Most commonly, they are used for static metadata extraction and service identification. The flexible definition allows custom extension of the core defined services to provide extra service type identification and metadata extraction. This is important due to the volume of custom applications that are used on networks.
NetWitness Platform has a set of core parsers that are defined by the system, and also has the ability to add additional parsers. Each parser is configurable in the Services Config View - General Tab. The Parser Configuration panel provides a way to enable or disable parsers to use on the Decoder in addition to limiting the metadata that the parser creates.
In addition, there are several types of custom configurable parsers:
- GeoIP2 or GeoIP – These parsers associate IP addresses with geographical locations. For new installations and upgrades, the GeoIP2 parser is enabled by default. Only one of these parsers can be enabled at a time. For more information on these parsers, see GeoIP2 and GeoIP Parsers.
- Search – This parser is user‐configured to generate metadata by scanning for pre‐defined keywords and regular expressions.
- FLEXPARSE (deprecated) – This is a generic parser definition language for extending the existing application protocol support of the Decoder. By default this parser is disabled (see Enable or Disable Lua and Flex Parsing Systems).
- Lua – This parser is defined using the Lua scripting language for extending the existing application protocol support of the Decoder.
- enVision – This application parser supports the Log Decoder and is configured to generate metadata by scanning log files.
- Snort® – This parser supports the payload detection capabilities of Snort IDS rules. Snort rules and configuration are added to the parsers/snort directory for Investigation and Decoder (see Snort Parsers).
In the Services Config view > Parsers tab, you can view deployed parsers on a Decoder, upload parsers, and delete deployed parsers. The user interface includes an Indicator if the parser originated from Live Services, installed through NetWitness Platform, or uploaded manually. Parsers can be added and removed while a Decoder is running without affecting capture.
In addition, you can download parsers using NetWitness Platform Live Services.
NetWitness Platform uses feeds to create metadata based on externally defined metadata values. A feed is a list of data that is compared to sessions as they are captured or processed. For each match, additional metadata is created. This data could identify and classify malicious IPs or incorporate additional information such as department and location based on internal network assignments. Some examples of feeds include threat feeds to identify BOTNets, DHCP mappings, or even active directory information such as physical location or logical department.
You can use the Live module in NetWitness Platform to obtain feeds from outside sources. "Live Content in NetWitness Platform" in the Live Services Management Guide provides an overview of the Live content management tool.
Within the NetWitness Platform user interface, you can view the list of currently deployed feeds, along with an indicator if a feed that originated from Live was installed through NetWitness Platform or manually. Feeds can be added, removed, and updated while a Decoder is running without affecting capture.
There is a Custom Feed wizard that allows creation and deployment of custom Decoder feeds based on deterministic logic that offers the meta keys specific to the selected Decoders and Log Decoders. Although the wizard guides users through the process to create both on-demand and recurring feeds, it is helpful to understand the form and content of a feed file when you create a feed.
NetWitness Platform has a Custom Feed wizard, which streamlines the task of creating and managing custom feeds, as well as populating the feeds to selected Decoders and Log Decoders. In addition, you can download existing feed files and edit the files, then edit the feed or create a new feed using the edited file.