NetWitness Investigate provides the data analysis capabilities in RSA NetWitness Platform. Using Investigate, analysts can examine packet, log, and endpoint data, and identify possible internal or external threats to security and the IP infrastructure.
Metadata, Meta Keys, Meta Values, and Meta Entities
RSA NetWitness Platform audits and monitors all traffic on a network. One type of service--a Decoder--ingests, parses, and stores the original packets captured on the network, logs forwarded by a device, and endpoint events seen by the endpoint agent.
The configured rules, parsers, and feeds on the Decoder create metadata that analysts can use to investigate the ingested logs, packets, and endpoint data. Another type of service, called a Concentrator, indexes and stores the metadata.
The metadata is in the form of a meta key and meta values for the key. For example, ip.src is a meta key, and an IP address that is the source of the traffic is tagged as ip.src. When you view data in Investigate, you see the meta key ip.src and all of the IP addresses (values) that are tagged with that key. Some meta keys are built-in and others may be custom keys defined by the administrator.
Meta entities are available in Version 11.1 and later. A meta entity is an alias that groups together the results from other meta keys. Meta entities organize similar meta keys into a single, easier to use, meta type. Some meta entities are already included by default, and the administrator can create custom meta entities. Analysts can use a meta entity in a query, a meta group, a column group, and a profile. Parallel coordinates visualizations do not support meta entities. Administrators can use meta entities to define a query prefix to apply to a user role and a user. The Decoder Configuration Guide provides additional information about creating meta entities and how they can be used in rules.
For example, the default Core database language includes distinct meta keys for IP source and IP destination. One of the built-in meta entities named ip.all represents the combined set of all IP sources and destinations.
Analysts usually query the Broker or Concentrator to discover threats. The Concentrator handles queries, only going to the Decoder when a full reconstruction of sessions or raw logs is required. ESA, Malware Analysis, and Reporting Engine also query the Concentrator, where they can quickly get all the pertinent metadata associated with an event and generate information about the event without having to query each Decoder. In some special cases, analysts may query a Decoder.
Triggers for an Investigation
These are a few examples of triggers for an investigation:
- You receive intelligence about a new active directory hack. Starting in the Events view, you use that intelligence to run a search across all of your raw Active Directory log data for the last 24 hours.
- You are asked by the SOC manager to find any Pokemon Go malware due to its popularity. Starting in the Navigate view, you craft a query to look for an HTTP session using a specific user agent related to the malware that your SOC manager found on a security blog.
- An incident responder escalates a ticket that shows some odd indicators related to a host. Starting in the Hosts view, you examine that host to find specific details.
- You are looking for the next zero day attack and start pivoting through network metadata in the Navigate view to find any abnormal automated sessions leaving the enterprise.
- You are asked by your SOC manager to find any information related to user jarvis, an employee who was just let go. Starting in the Investigate > Entities > Users tab (UEBA) you can filter for that username to make sure there is no longer any activity for that user and see if that user deviated from their typical behavior prior to being let go.
A phishing attack detected has an associated attachment, and you want to know what devices in your environment have seen that file by searching for the file hash in Investigate > Files.
A malicious file has been automatically found in your environment, and you want to review the static and dynamic analysis done on that file along with how many systems it has been transmitted to or from. Starting in Investigate > Malware Analysis you can see the analysis results.
Workflow of an Investigation
Analysts can investigate data captured by NetWitness Platform, and deep dive from information on a NetWitness Platform dashboard, a NetWitness Respond incident or alert, a report created by the NetWitness Platform Reporting Engine, or a third-party application. During the course of an investigation, analysts can move seamlessly between the views in Investigate: the Navigate view, the Events view, the Legacy Events view, the Hosts view, the Files view, the Entities view, and the Malware Analysis view. This figure illustrates the NetWitness Investigate submenus.
You can access each view from the Investigate submenu and from other Investigate views. You can also go directly into an Investigate view from NetWitness Respond, and go directly from NetWitness Investigate to NetWitness Respond and to standalone NetWitness Endpoint. The use case determines the starting point for your investigation. Every situation is unique in terms of the types of information you are attempting to find. Many investigations start in one view, and end in a different view as you learn something and then need to follow that result to a different line of questioning. The following table provides general guidance on the starting view for different use cases.
Focus on Metadata, Query, and Time
The following figure depicts the workflow for an investigation with focus on metadata, a query, and time range.
Analysts use Investigate to hunt for events that drive the incident response workflow and to do strategic analysis after another tool has generated an event. Beginning in the Navigate view, Events view, or Legacy Events view:
- You start by executing a query on a service for a specific time range, then filter results to get a subset of events, reconstruct or analyze an event, and repeat the process to reconstruct or analyze another event. Built-in profiles, meta groups, and column groups provide a good starting point. For example, you can choose the RSA Email Analysis profile to see only metadata that is useful when investigating email risks.
- When you encounter an event that bears a closer look, you view the context around the event, and decide whether to create an incident or add the event to an incident. If you decide not to add the event to an incident, you run another query to gain further insight, which starts again at the beginning of the workflow.
- If you notice suspicious activity or files on a specific host in the network, you can gather additional information about the host and files found on the host in the Hosts and Files view, or in a standalone NetWitness Endpoint server.
- If you find a file or event that potentially contains malware, you can do a Malware Analysis scan of the file or you can open Malware Analysis and start a scan of the service on which the event was seen.
Here is one simple use case: If there is a concern regarding suspicious traffic with foreign countries, the Destination Country meta key reveals all destinations and the frequency of the contact. Drilling into those values yields the specifics of the traffic, such as the IP address of the originator and the recipient. Checking other metadata can expose the nature of attachments exchanged between the two IP addresses. When suspect IP addresses are identified, looking at the addresses in the Navigate view or Events view with a broader time range can provide clues about what happened before and after the event being investigated.
Another use case is to investigate an alert to discover a malicious insider in the network who is exfiltrating intellectual property or other sensitive data from a specific IP address. The investigation begins with this meta value: Upload without change request followed by download alert. Start in the Navigate view by filtering the values to the IP address during the time range in which the alert was generated. Alerts metadata in the Navigate view show risk indicators as meta values, and you can click on different meta values to reconstruct the event. Next extract files and examine the files to understand what happened. With this information, you can filter on the same IP address and broaden the time range to see activities before and after the event.
Focus on Respond View Incidents and Alerts
An analyst who is working on an incident or an alert in Respond can open the incident in Investigate (Navigate view) to do a deeper analysis of the event or alert.
- The workflow to respond to an incident typically begins in the Respond view, where the analyst who is investigating an incident needs to gather intelligence about the incident in Investigate. You can hover over an underlined entity in an incident or alert, such as an IP address, and then select the action Pivot to Investigate > Navigate. The Navigate view opens and is filtered for the selected entity. After you launch an investigation from the Respond view, defined meta keys are queried and the captured packets, logs, and endpoint events are displayed in the Navigate view.
- If you find events that are relevant to the incident, you can add the events to the incident in Respond. You can also create a new incident in Respond based on one or more events found in Investigate.
- From the Incident Details view Indicators panel in Respond, you can open the Events view to get a better understanding of an indicator event.
NetWitness Investigate Views
This section provides a brief description and example of the Navigate view, Events view, and Legacy Events view, and views that provide additional context for data found – the Context Lookup panel and the Event Reconstruction view.
Refer to these guides for information about other Investigate views:
- The NetWitness Endpoint User Guide covers features and functions of the Hosts view and Files view.
- The NetWitness UEBA User Guide covers features and functions of the Entities (formerly Users) view.
- The Malware Analysis User Guide covers features and functions of the Malware Analysis view.
The Navigate view provides the capability to drill into and query contents of captured packets, logs, and endpoint events on a Broker, Concentrator, or Decoder (though investigating a Decoder is not typical).
- When you select a service and a time range, the defined meta keys for that service are queried, and values are returned along with the number of events. Clicking a value filters out the other values, providing a view into a more focused set of data. This is called drilling into the data.
- For certain configured meta keys, such as IP address, or hostname, you can see additional context information around a value using the Context Hub. The additional context may include incidents, alerts, and other sources where the value was mentioned. This additional context is retrieved from sources other than the original data providing the analyst further perspective around the event. As an example of context, the associated incidents and alerts convey if this event or a similar one has been seen or previously worked on before. Any lists that match the metadata can help indicate if the metadata relates to a known blacklist of known adversaries or another analysts list of users they have found to be against typical usage policies.
- The Navigate view also provides a sequential visualization of the data in a timeline. You can zoom in on a selected time period.
This figure illustrates the Navigate view.
The default workflow for analysts interacting with events is optimized to limit the need to transition from one view to another. By combining capabilities (highlights spelled out further in this document) that were previously in two distinct workflows, referred to Event Analysis and Events, the analyst now has a single workflow for analyzing events. By default the previous workflow is not in the Investigate menu, but an administrator can re-enable it if they desire a transitional period for existing analysts.. Events are listed in order by time.
The Events list shows the raw data for events, which you can sort and filter. You can also apply column groups to control which columns are displayed and their arrangement across the view. You can use Query profiles to apply a built-in or custom column group and a prequery in this view.
- The related metadata for results in the Events list is shown in the Event Meta panel. Analysts reviewing the metadata can change the order of the metadata to better track down what they are looking for. The items in the list of metadata can optionally be grouped by the sequence they were generated or alphabetically.
- Clicking an event opens a reconstruction of the event. Different reconstructions are available (packets, text, files) with helpful cues to identify points of interest, such as interesting bytes, file types, and encoded data. Email and web reconstructions open in a new Legacy Events view window.
- For certain configured meta keys, such as IP address, or hostname, you can search for additional context information around a value using the Context Hub. The additional context may include incidents, alerts, and other sources where the value was mentioned.
- You can pivot to standalone Endpoint, look up in Live and do other internal lookups, and do external lookups. External lookups allow you to search the internet for meta values with which you interacted, determine passive DNS information related to an IP address, ascertain if a URL is blacklisted, and other third-party context integrations.
- When viewing files, you can export files in a zip archive to your local file system.
- You can download logs from the Text view, and export packets from the Packet view. You can download multiple events from the Events list.
This figure is an example of the Events view with a packet reconstruction open in the right panel. The Events list is visible in the left panel.
Legacy Events View
The Legacy Events view was the original Events view (11.0 to 11.3.x.x). The Legacy Events is no longer needed in Version 11.4, and it is hidden unless the administrator enables it as described under "Configure Investigation Settings" in the System Configuration Guide. The Legacy Events view provides a view of packet, log, and endpoint events in list form so that you can view events sequentially and reconstruct events safely.
- You can open the Legacy Events view for a meta value that you see in the Navigate view.
- For analysts without sufficient privilege to navigate a service, the Legacy Events view is a standalone investigation view in which analysts can access a list of network, log, and endpoint events from a NetWitness Platform Core service without having to drill down through metadata first.
- The Legacy Events view presents event information in three standard forms, a simple list of events, a detailed listing of events, and a log view.
- For certain configured meta keys, such as IP address, or hostname, you can see additional context information around a value using the Context Hub. The additional context may include incidents, alerts, and other sources where the value was mentioned.
- You can export events and associated files, and create an incident from an event.
This figure illustrates the Legacy Events view.
Contextual Information for an Event
In the Navigate view, Events view, and Legacy Events view, the Context Lookup panel shows details about elements associated with an event in the Context Hub for these meta types: IP Address, User, Host, Domain, MAC Address, Filename, and File hash. In addition, you can right-click all meta keys except time to see additional context.
- You can interact with the elements of an event to get further insight including related incidents, alerts, custom lists, Archer assets, active directory details, and NetWitness Endpoint IIOCs.
- You can click on a data point to go to the Navigate view and Legacy Events view.
The following figure shows the Context Lookup panel in the Events view.
The following figure shows the Context Lookup panel in the Legacy Events view.
Multiple views offer the ability to reconstruct an event. When you discover an event that merits additional investigation, you can reconstruct an event safely in a form similar to its native form. The rendering of events restricts the use of dynamic or active code that might be contained in the event to limit any adverse outcome to your system or browser. Cache is used to improve performance when viewing previously viewed events. Each analyst has a separate cache of reconstruction data, and you can only access reconstructed events in your own cache.
The Events view presents an interactive event reconstruction, which includes raw data, meta keys, and values. This figure is an example of a reconstruction in the Events view.
In the Events view reconstruction, you can:
- Reconstruct an event using different methods to suit the type of data: metadata, text, hexadecimal, packets, and files.
- Understand more about highlighted information in the headers and payloads.
- View decoded and encoded payloads and see common file signatures.
- Search for locations of meta keys or values in the reconstruction.
- Export events and files.
The Event Reconstruction in the Legacy Events view presents the raw data and the meta keys and meta values for an event in a list form. This figure is an example of the Event Reconstruction.
In this Event Reconstruction, you can:
Page through the reconstruction to view the next event in this form.
Reconstruct events using different methods to suit the type of data: metadata, text, hexadecimal, packets, web, mail, files, or the best reconstruction selected automatically.
- Export packet capture files, extract files, and export the meta values for the event.