|Due to problems saving this topic as a PDF, please use the following link if you require a PDF of this content:|
Hunting Guide PDF
NetWitness is an evolution of the NetWitness NextGen security product, formerly known as Security Analytics. The platform ingests network traffic and logs, applies several layers of logic against the data, stores the values in a custom time-based database, and presents the metadata to the analyst in a unified view. When integrated with ECAT, a host based memory forensics tool, metadata about host activities is generated and presented in the same view, giving the analyst an unparalleled view into the state of the network. In this guide we will be discussing tactics and procedures for investigating the packet dataset for malicious activity.
NetWitness is not a typical network traffic based sensor, it is not an IDS/IPS or Netflow device, although some of its more basic capabilities could provide some overlap. Metadata is generated to describe a technical aspect or behavior within a network session. A session is defined as one or two related stream(s) of traffic with a requestor and, usually, a responder. These sessions are ordered by capture time and as such time is the first WHERE clause applied to the database when beginning an investigation. Knowing how the data is collected and ordered is integral to understanding how to hunt in NetWitness.
Metadata in NetWitness should be considered indicators of an activity, not signatures like those used by traditional IDS/IPS and as such should be handled differently. The logic contained in the NetWitness parsers is far more versatile than your typical regex based signatures. The parsers, feeds and application rules that process traffic generate metadata about the structure of the data and extract values from the individual sessions that can be searched for efficiently. This differs from traditional IDS/IPS solutions in that it is possible to find new unknown malicious activity compared to only finding previously identified malicious activity. Signature-like parsers are also included, but because the parser engine is using a common scripting language, Lua, more complex logic can be used to determine a match, giving a far lower false-positive rate when used in this manner. This guide focuses on hunting for new unknown malicious activity using the content provided by the RSA Live content management system and generally does not include an overview of signature-like parsers.
Hunting within the NetWitness dataset is accomplished by analyzing intrusions, reverse engineering malware, analyzing traffic generated by malware and other attacks, then selecting metadata generated by NetWitness based on this type of behavior. The RSA IR team has conducted many investigations since being formed in 2012 and has created content and tactics for the platform that allow an analyst to quickly navigate the dataset by combining many aspects of behavior into a single piece of metadata. This cuts down on the number of drills needed to find the sessions with the desired behavior, enhancing performance of the platform and reducing the effort needed to find malicious behavior. This has allowed the IR team to discover incidents without any prior knowledge or notification that the organization was under a targeted attack. The IR team has also used these methodologies and content to discovery many incidents where the attacker wasn’t even using malware, but authenticated access, also called Living off the “LANd”.
The unprecedented view into network traffic provided by NetWitness is most effective for Incident Response capabilities, but can also be used to validate the appropriate enforcement of your security policies and/or uncover areas where these policies and procedures may require improvement. This guide is intended for analysts who want to uncover new malicious activity and not simply react to alerts based on known threats.
The Hunting pack is designed to allow you to quickly hunt for indicators of compromise or anomalous network activity by dissecting packet traffic within the NetWitness Suite and populating specific meta keys with natural language values for investigation.
The Hunting pack consists of the following separate pieces:
- A set of meta keys that are populated with the indicators
- Imports of meta groups, which provide a view to the analyst of relevant combinations of meta data
- A set of Lua parsers to dissect the network sessions from common protocols used by an attacker
- The Investigation Feed and the RSA FirstWatch SSL Blacklist feed.
- Hunting-related RSA NetWitness reports
- Hunting-related RSA NetWitness rules
- Webshell Detected ESA rule: This rule indicates that 3 webshells have been detected through communication between the same IP source and destination pair within a 10 minute time window. More details are available in the RSA ESA Rules topic.
Deploying the Hunting Pack
You can deploy all of the items in the Hunting Pack through Live.
Note the following:
- For deployments prior to 10.6.2, you will also need to configure a set of new meta keys: netname, direction, ioc, boc, eoc, analysis.service, analysis.session, analysis.file. For details, see Meta Keys.
- The trafflic_flow Lua parser may be deployed to a Log Decoder, but this is not currently supported through Live. In the Traffic Flow Lua Parser documentation, https://community.rsa.com/docs/DOC-44948, see the section Deploy to Log Decoders.
- If you are in an environment where you cannot Deploy, you should create a resource package (select > Create) to download a ZIP archive that you can use. Do not use the button, as this does not work for bundles.
To deploy the Hunting pack, depending on your version, see:
The meta keys that are populated as a result of the Lua parser deployment that make up the Hunting content pack are as follows. These are available without additional configuration in version 10.6.2 and higher of the NetWitness Suite. If you are deploying the content pack to a version prior to this, then see Appendix: Hunting Content Pack Meta Keys for instructions to enable them.
NetWitness offers the analyst a method to customize the metadata views and groups that are displayed while conducting an investigation. Before beginning to hunt, the first items to set up are metadata groups. RSA provides a ZIP of files that contain Meta groups for incident response hunting. These files are available as a ZIP archive in the Downloads space on RSA Link at the following URL: https://community.rsa.com/docs/DOC-60112.
For deployment of the meta groups, see the product documentation Import a Meta Group under the topic Investigation: Manage User-Defined Meta Groups. By default, the meta keys are in the ‘Close’ state. You may change to ‘Open’ view state by default for each key, depending on your needs and performance considerations.
You may deploy the Hunting pack Lua parsers from Live. Select the parsers listed below within the Live Search UI and choose to go through the process of deployment or subscription to a Decoder.
List of Lua Parsers in the Hunting Pack
Lua Parser Options Files
The following Lua Parsers currently have options files associated with them:
Note the following:
- If you deploy the options file, it can be found in the same directory as parsers:
- The parser is not dependent upon the options file. The parser will load and run even in the absence of the options file. The options file is only required if you need to change the default settings.
- If you do not have an options file (or if your options file is invalid), the parser uses the default settings.
RSA NetWitness Platform Reports
RSA provides two reports as part of the Hunting Pack:
Hunting Summary Report: This report displays a summary of the events that have been categorized according to the following meta keys.
Hunting Detail Report: This report displays events that have been categorized according to the following meta keys with added contextual evidence to assist an analyst.
These reports are based on events that have been categorized according to the following meta keys:
- Indicators of Compromise
- Behaviors of Compromise
- Enablers of Compromise
- Service Analysis
- Session Analysis
- File Analysis
These keys are described in the Meta Keys section.
RSA NetWitness Platform Rules
The two Hunting Pack reports are dependent on the following rules.
The Hunting Summary Report is dependent upon these rules:
- Behaviors of Compromise: Designated for suspect or nefarious behavior outside the standard signature-based detection. This rule displays output when the meta key, Behaviors of Compromise, is populated.
- Enablers of Compromise: Instances of poor information or operational security. Post-mortem often ties these to the root cause. This rule displays output when the meta key, Enablers of Compromise, is populated.
- File Analysis: A large inspection library that highlights file characteristics and anomalies. This rule displays output when the meta key, File Analysis, is populated.
- Indicators of Compromise: Possible intrusions into the network that can be identified through malware signatures or IPs and domains associated with command and control campaigns. This rule displays output when the meta key, Indicators of Compromise, is populated.
- Service Analysis: Core application protocols identification and inspection. This rule displays output when the meta key, Service Analysis, is populated.
- Session Analysis: A large inspection library that highlights file characteristics and anomalies. This rule displays output when the meta key, File Analysis, is populated.
The Hunting Details Report is dependent on these rules:
- Behaviors of Compromise Detail: Additional context (compared to Behaviors of Compromise rule) is provided to an analyst by grouping with additional meta keys of Service Type and Device Type.
- Enablers of Compromise Detail: Additional context (compared to Enablers of Compromise rule) is provided to an analyst by grouping with additional meta keys of Service Type and Device Type.
- File Analysis Detail: Additional context (compared to File Analysis rule) is provided to an analyst by grouping with the additional meta key of Filename.
- Indicators of Compromise Detail: Additional context (compared to Indicators of Compromise rule) is provided to an analyst by grouping with additional meta keys of Service Type and Device Type.
- Service Analysis Detail: Additional context (compared to Service Analysis rule) is provided to an analyst by grouping with additional meta keys of Service Type and Alias Host.
- Session Analysis Detail: Additional context (compared to Session Analysis rule) is provided to an analyst by grouping with additional meta keys of Service Type and Alias Host.
It is important to understand how network traffic is processed by NetWitness and displayed to the user. Figure 1 shows how the Decoder service captures packets and copies them into memory in what are called ‘pages’. The first pool a frame lands in when it is captured is the packet capture pool. Here sessions are either begun or packets added to an existing session in the Assembler. NetWitness is IPv4 and IPv6 aware and will mark the first frame in a TCP session that contains the TCP SYN flag as the Request and the other end as Response. Non-TCP based IP protocols or continuation traffic’s directionality is determined by several criteria.
- Client talks first
- Server usually provides more data
- Server usually has a lower port, if available
- Server should be a non-RFC1918 IP
- Organizations usually use lower IP octets for static IP addresses and servers
These considerations are weighted and can be adjusted by changing the values in assembler.voting.weights within the Explorer interface.
When a session is begun in the Assembler two timers begin. One is counting seconds since the session has been started and after 60 seconds (SA default) the session will be declared over, parsed and written to disk. The second timer is a byte timer, after 32 MB (SA default) a session will be declared over, parsed and written to disk. There are some edge cases where extremely low bandwidth and long lived sessions will stay in the Assembler for the entire duration of the session and will be presented end to end with a lifetime value of over 60 seconds.
If you have ever used NetWitness for a length of time, you will quickly realize networks are noisy. There are retransmissions, single sided sessions, zero payload sessions, and Peer-to-Peer communications that make analyzing a dataset more difficult. When analyzing a dataset, you have to start with a direction. Do you want to view inside-to-outside, outside-to-inside, or inside-to-inside? The traffic_flow.lua parser makes this determination based on options set in the traffic_flow_options.lua file on the decoder. For details, see the Traffic Flow Lua Parser topic on RSA Link.
This defines RFC1918 IP address space as well as other non-routable blocks of IPs used to determine direction. It is advised that an organization modifies the provided options file with internal networks and their names as well as any non-RFC1918 IP space used by the organization, for example interesting traffic ACL’s for LAN-to-LAN IPSEC tunnels.
The following table shows metadata stored in Direction that is used for traffic flow by default without modifying the traffic_flow_options.lua file.
Session Characteristics Meta Category
The Session Characteristics Meta Category extends this logic by examining technical aspects of the captured sessions. It checks the number of streams, if any payload was transmitted in those streams, the lifetime of the session, the size and ratio of transmitted vs. received data and also combines some of this logic to give the analyst a clearer view into their network. The table below describes the Session Characteristics meta category—these meta keys are populated by the session_analysis Lua parser.