Investigate: How NetWitness Investigate Works

Document created by RSA Information Design and Development on Sep 18, 2017Last modified by RSA Information Design and Development on Apr 25, 2019
Version 16Show Document
  • View in full screen mode

NetWitness Investigate provides the data analysis capabilities in RSA NetWitness® Platform, so that analysts can analyze packet, log, and endpoint data, and identify possible internal or external threats to security and the IP infrastructure.

Note: In Version 11.1 and later, the Hosts and Files views provide a view into endpoint data. Earlier versions offer access to endpoint data using a standalone NetWitness Endpoint server.

Metadata, Meta Keys, Meta Values, and Meta Entities

RSA NetWitness Platform audits and monitors all traffic on a network. One type of service--a Decoder--ingests, parses, and stores the packets, logs, and endpoint data traversing the network.

The configured parsers and feeds on the Decoder create metadata that analysts can use to investigate the ingested logs and packets. Another type of service, called a Concentrator, indexes and stores the metadata.

The metadata is in the form of a meta key and meta values for the key. For example, ip.src is a meta key, and an IP address that is the source of the traffic is tagged as ip.src. When you view data in Investigate, you see the meta key ip.src and all of the IP addresses (values) that are tagged with that key. Some meta keys are built-in and others may be custom keys defined by the administrator.

Meta entities are available in Version 11.1 and later. A meta entity is an alias that groups together the results from other meta keys. Meta entities organize similar meta keys into a single, easier to use, meta type. Some meta entities are already included by default, and the administrator can create custom meta entities. Analysts can use a meta entity in a query, a meta group, a column group, and a profile. Parallel coordinates visualizations do not support meta entities. Administrators can use meta entities to define a query prefix to apply to a user role and a user. The Decoder Configuration Guide provides additional information about creating meta entities and how they can be used in rules.

Note: Meta entities need to be configured on all upstream Concentrators. If any Concentrator does not have a meta entity configured, that meta entity will be empty when you query the Broker.

For example, the default Core database language includes distinct meta keys for IP source and IP destination. One of the built-in meta entities named ip.all represents the combined set of all IP sources and destinations.

Analysts usually query the Concentrator to discover threats. The Concentrator handles queries, only going to the Decoder when a full reconstruction of sessions or raw logs is required. ESA, Malware Analysis, and Reporting Engine also query the Concentrator, where they can quickly get all the pertinent metadata associated with an event and generate information on the event without having to go to each Decoder. In some special cases, analysts may query a Decoder.

Triggers for an Investigation

These are a few examples of triggers for an investigation:

  • You receive intelligence about a new active directory hack. Starting in the Events view, you use that intelligence to run a search across all of your raw Active Directory log data for the last 24 hours.
  • You are asked by the SOC manager to find any Pokemon Go malware due to its popularity. Starting in the Navigate view, you craft a query to look for an HTTP session using a specific user agent related to the malware he found on a security blog.
  • An incident responder escalates a ticket that shows some odd indicators related to a host. Starting in the Hosts view, you examine that host to find specific details.
  • You are looking for the next zero day attack and start pivoting through network metadata in the Navigate view to find any abnormal automated sessions leaving the enterprise.
  • You are asked by your SOC manager to find any information related to user jarvis, an employee who was just let go. Starting in the Hosts view, you query against the past week for that username.

Workflow of an Investigation

Analysts can investigate data captured by NetWitness Platform, and deep dive from information on a NetWitness Platform dashboard, a NetWitness Respond incident or alert, a report created by the NetWitness Platform Reporting Engine, or a third-party application. During the course of an investigation, analysts can move seamlessly between the views in Investigate: the Navigate view, the Events view, the Event Analysis view, the Hosts view, the Files view, the Users view, and the Malware Analysis view. This figure illustrates the NetWitness Investigate submenus.

NetWitness Investigate Submenus

Note: The Files and Hosts views are available in Version 11.1 and later. The Users view is available in Version 11.2 and later. Specific user roles and permissions are required for a user to conduct investigations and malware analysis in NetWitness Platform. If you cannot perform an analysis task or see a view, the administrator may need to adjust the roles and permissions configured for you.

You can access each view from the Investigate submenu and from other Investigate views. You can also go directly into an Investigate view from NetWitness Respond, and go directly from NetWitness Investigate to NetWitness Respond and standalone NetWitness Endpoint. Your use case determines the starting point for your investigation. This table provides general guidance on the starting view for different use cases.

Go to...Focus
Navigate viewAll meta keys and meta values for logs, endpoints, and packets are grouped by meta key. You can pivot through the data to refine results, then go to the Events view or Event Analysis view, or look up in Malware Analysis or Live. This is the default NetWitness Investigate view. (See Investigating Metadata in the Navigate View.)
Events viewEvents are listed in order by time. You can view the raw event and related metadata, view a reconstruction, and download events and files. You can go to the Event Analysis view. (See Examining Raw Events in the Events View.)
Event Analysis viewEvents are listed in order by time. You can view the raw event and related metadata, view a reconstruction that offers helpful cues to identify points of interest in a reconstruction, such as interesting bytes, file types, and encoded data. You can pivot to standalone Endpoint, look up in Live, and do external lookups. External lookups allow you to search the internet for meta values with which you interacted, determine passive DNS information related to an IP address, ascertain if a URL is blacklisted, and other third-party context integrations. (See Analyzing Raw Events and Metadata in the Event Analysis View)
(Version 11.1 and later) Hosts viewHosts on which the NetWitness Endpoint Insights Agents are running are listed. For every host, you can view processes, drivers, DLLs, files (executables), services, anomalies, and autoruns that are running, and information related to logged-in users. From the Hosts view, you can go to the Navigate, Event Analysis, and Users views. (See the NetWitness Endpoint User Guide.)
(Version 11.1 and later) Files viewUnique files such as PE, Macho, and ELF in your deployment are listed. For each file, you can view details such as file name, reputation status, file status, risk score, signature, checksum, and others. From the Files view, you can go to the Navigate and Event Analysis views. (See the NetWitness Endpoint User Guide.)
Malware Analysis viewIf you are running a Malware Analysis appliance, you can scan files and see results of four types of analysis: network, static, community, and sandbox. If a file is malware, you can go to the Hosts view to see which hosts downloaded the file. (See the Malware Analysis User Guide.)

(Version 11.2 and later) Users view

Provides visibility into risky user behaviors across your enterprise using NetWitness UEBA. You can view a list of high-risk users and a summary of the top alerts for risky behavior for your environment, and then select a user or an alert and view details about the risky behavior, and a timeline during which the behaviors occurred. NetWitness Platform users assigned the Administrators or UEBA Analysts role have access to this view. (See the NetWitness UEBA User Guide..

Every situation is unique in terms of the types of information the analyst is attempting to find. Many investigations start in one view, and end in a different view as the analyst learns something and then needs to follow that result to a different line of questioning. This figure shows the high-level capabilities of NetWitness Investigate. In the top box are all the possible starting points, and the lower box shows the tasks that you can accomplish from different starting points.

the high-level workflow for NetWitness Investigate

Focus on Metadata, Query, and Time

The following figure depicts the workflow for an investigation with focus on metadata, a query, and time range.

the workflow for investigating a network or log event

Analysts use NetWitness Investigate to hunt for events that drive the incident response workflow and to do strategic analysis after another tool has generated an event. Beginning in the Navigate view, Events view, or Event Analysis view:

  • You start by executing a query on a service for a specific time range, then filter results to get a subset of events, reconstruct or analyze an event, and repeat the process to reconstruct or analyze another event. Built in profiles, meta groups, and column groups provide a good starting point. For example, you can choose the RSA Email Analysis profile to see only metadata that is useful when investigating email risks.
  • When you encounter an event that bears a closer look, you view the context around the event, and decide whether to create an incident or add the event to an incident. If you decide not to add the event to an incident, you run another query to gain further insight, which starts again at the beginning of the workflow.
  • If you notice suspicious activity or files on a specific host in the network, you can gather additional information about the host and files found on the host in the Hosts and Files view, or in a standalone NetWitness Endpoint server.
  • If you find a file or event that potentially contains malware, you can do a Malware Analysis scan of the file or you can open Malware Analysis and start a scan of the service on which the event was seen.

Here is one simple use case: If there is a concern regarding suspicious traffic with foreign countries, the Destination Country meta key reveals all destinations and the frequency of the contact. Drilling into those values yields the specifics of the traffic, such as the IP address of the originator and the recipient. Checking other metadata can expose the nature of attachments exchanged between the two IP addresses. When suspect IP addresses are identified, looking at the addresses in the Navigate view or Event Analysis view with a broader time range can provide clues about what happened before and after the event being investigated.

Another use case is to investigate an alert to discover a malicious insider in the network who is exfiltrating intellectual property or other sensitive data from a specific IP address. The investigation begins with an Upload without change request followed by download alert. Start in the Navigate view by filtering the values to the IP address during the time range in which the alert was generated. Alerts metadata in the Navigate view show risk indicators as meta values and you can click on different meta values to reconstruct the event. Next extract files and examine the files to understand what happened. With this information, you can filter on the same IP address and broaden the time range to see activities before and after the event.

Focus on NetWitness Respond Incidents and Alerts

An analyst who is working on an incident or an alert in NetWitness Respond can open the incident in NetWitness Investigate (Navigate view) to do a deeper analysis of the event or alert.

  • The workflow to respond to an incident typically begins in the Respond view, where the analyst who is investigating an incident needs to gather intelligence about the incident in NetWitness Investigate. You can hover over an underlined entity in an incident or alert, such as an IP address, and then select the action Pivot to Investigate > Navigate. The Navigate view opens and is filtered for the selected entity. After you launch an investigation from NetWitness Respond, defined meta keys are queried and the content of captured packets, logs, and endpoint events is displayed in the Navigate view.
  • If you find events that are relevant to the incident, you can add the events to the incident in Respond. You can also create a new incident in Respond based on one or more events found in Investigate.
  • (Version 11.2 and later) From the Incident Details view Indicators panel in Respond, you can open the Event Analysis view to get a better understanding of an indicator event.

NetWitness Investigate Views

This section provides a brief description and example of the Navigate view, Events view, and Event Analysis view, and views that provide additional context for data found--the Context Lookup panel and the Event Reconstruction view.

Refer to these guides for information about other Investigate views:

  • The NetWitness Endpoint User Guide covers features and functions of the Hosts view and Files view.
  • The NetWitness UEBA User Guide covers features and functions of the Users view.
  • The Malware Analysis User Guide covers features and functions of the Malware Analysis view.

Navigate View

The Navigate view provides the capability to drill into and query contents of captured packets, logs, and endpoint events on a Broker, Concentrator, or Decoder (though investigating a Decoder is not typical).

  • When you select a service, the defined meta keys for that service are queried, and values are returned along with the number of events. Clicking a value filters out the other value,s providing a view into a more focused set of data. This is called drilling into the data.
  • For certain configured meta keys, such as IP address, or hostname, you can see additional context information around a value using the Context Hub. The additional context may include incidents, alerts, and other sources where the value was mentioned.
  • The Navigate view also provides a sequential visualization of the data in a timeline. You can zoom in on a selected time period.

This figure illustrates the Navigate view.

an example of the Navigate view

Events View

The Events view provides a view of packet, log, and endpoint events in list form so that you can view events sequentially and reconstruct events safely.

  • You can open the Events view for a meta value that you see in the Navigate view.
  • For analysts without sufficient privilege to navigate a service, the Events view is a standalone investigation view in which analysts can access a list of network, log, and endpoint events from a NetWitness Platform Core service without having to drill down through metadata first.
  • The Events view presents event information in three standard forms, a simple list of events, a detailed listing of events, and a log view.
  • For certain configured meta keys, such as IP address, or hostname, you can see additional context information around a value using the Context Hub. The additional context may include incidents, alerts, and other sources where the value was mentioned.
  • You can export events and associated files, and create an incident from an event.

This figure illustrates the Events view.

example of the Events view

Event Analysis View

The Event Analysis view is an interactive tool to help analysts see the packets, text, or files in an event with visual cues to highlight certain types of information. Depending on the type of reconstruction different information is relevant.

  • For certain configured meta keys, such as IP address, or hostname, you can search for additional context information around a value using the Context Hub. The additional context may include incidents, alerts, and other sources where the value was mentioned.
  • When viewing files, you can export files in a zip archive to your local file system.
  • You can download logs from the Text view, and export packets from the Packet view.

This figure is an example of the Event Analysis view.

an example of the Event Analysis view

Contextual Information for an Event

In the Navigate view, Events view, and Event Analysis view (Version 11.2 and later), the Context Lookup panel shows details about elements associated with an event (IP Address, User, Host, Domain, MAC Address, Filename, and File hash) in the Context Hub. In addition, you can right-click all meta keys except time to see additional context.

  • You can interact with the elements of an event to get further insight including related incidents, alerts, custom lists, Archer assets, active directory details, and NetWitness Endpoint IIOCs.
  • You can click on a data point to go to the Navigate view.

Note: Archer assets and active directory details are available in the Event Analysis view context lookup. Endpoint context lookup is available for NetWitness Endpoint or later hosts, but not for the NetWitness Endpoint 11.1 hosts.

The following figures show the Context Lookup panel in the Events view and the Event Analysis view.

example of the Context Lookup panel open in the Events view

example of the Context Lookup panel in the Event Analysis view

Event Reconstruction

Three NetWitness Investigate views offer the ability to reconstruct an event: Navigate view, Events view and Event Analysis view. When you discover an event that merits additional investigation, you can reconstruct an event safely in a form similar to its native form. The rendering of events restricts the use of dynamic or active code that might be contained in the event to limit any adverse outcome to your system or browser. Cache is used to improve performance when viewing previously viewed events. Each analyst has a separate cache of reconstruction data, and you can only access reconstructed events in your own cache.

The Event Reconstruction in the Events view presents the raw data and the meta keys and meta values for an event in a list form. This figure is an example of the Event Reconstruction.

the Event Reconstruction view

In this Event Reconstruction, you can:

  • Page through the reconstruction to view the next event in this form.
  • Reconstruct events using different methods to suit the type of data: metadata, text, hexadecimal, packets, web, mail, files, or the best reconstruction selected automatically.
  • Export packet capture files, extract files, and export the meta values for the event.

The Event Analysis view presents an interactive event reconstruction, which includes raw data, meta keys, and values. This figure is an example of a reconstruction in the Event Analysis view.

the Event Analysis view reconstruction

In the Event Analysis view reconstruction, you can:

  • Reconstruct an event using different methods to suit the type of data: metadata, text, hexadecimal, packets, and files.
  • Understand more about highlighted information in the headers and payloads.
  • View decoded and encoded payloads and see common file signatures.
  • Search for locations of meta keys or values in the reconstruction.
  • Export events and files.

You are here
Table of Contents > How NetWitness Investigate Works